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Research  partially  supported  by  this  grant  culminated  in  the  submission  of  twenty  eight  new 
research  papers  twenty  three  of  which  have  now  been  accepted  for  publication.  A  detailed  descrip¬ 
tion  of  these  works  is  provided  below.  Four  Ph.D.  students  (Xin  Liu,  Dominik  Reinhold,  Jiang 
Chen  and  Xuan  Wang)  completed  their  dissertation  under  the  direction  of  the  PI.  The  topics  of 
their  study  are  outlined  below.  Three  other  Ph.D.  students  (Abhishek  Pal  Majumdar,  Ruoyu  Wu 
and  Eric  Friedlander)  continued  work  on  their  dissertations.  Postdoctoral  fellows  C.Y.Lee,  X.Song 
and  S.Saha  were  partially  supported  by  this  grant.  Invited  presentations  that  were  given  during 
this  period  are  also  listed. 


(I)  Large  Deviation  Properties  of  Weakly  Interacting  Processes  via  Weak  Convergence 
Methods  [14].  Collections  of  weakly  interacting  random  processes  have  long  been  of  interest  in 
statistical  physics,  and  more  recently  have  appeared  in  problems  of  engineering  and  operations 
research.  A  simple  but  important  example  of  such  a  collection  is  a  group  of  particles,  each  of  which 
evolves  according  to  the  solution  of  an  Ito  type  stochastic  differential  equation  (SDE).  All  particles 
have  the  same  functional  form  for  the  drift  and  diffusion  coefficients.  The  coefficients  of  particle  i 
are,  as  usual,  allowed  to  depend  on  the  current  state  of  particle  i,  but  also  depend  on  the  current 
empirical  distribution  of  all  particle  locations.  When  the  number  of  particles  is  large  the  contribu¬ 
tion  of  any  given  particle  to  the  empirical  distribution  is  small,  and  in  this  sense  the  interaction 
between  any  two  particles  is  considered  weak.  For  various  reasons,  including  model  simplification 
and  approximation,  one  may  consider  a  functional  law  of  large  numbers  (LLN)  limit  as  the  number 
of  particles  tends  to  infinity.  The  limit  behavior  of  a  single  particle  (under  assumptions  which 
guarantee  that  all  particles  are  in  some  sense  exchangeable)  can  be  described  by  a  two  component 


1 


Markov  process.  One  component  corresponds  to  the  state  of  a  typical  particle,  while  the  second 
corresponds  to  the  limit  of  the  empirical  measures.  Again  using  that  all  particles  are  exchangeable, 
under  appropriate  conditions  one  can  show  that  the  second  component  coincides  with  the  distri¬ 
bution  of  the  particle  component.  The  limit  process,  which  typically  has  an  infinite  dimensional 
state,  is  sometimes  referred  to  as  a  nonlinear  diffusion.  Because  the  particle’s  own  distribution 
appears  in  the  state  dynamics,  the  partial  differential  equations  that  characterize  expected  values 
and  densities  associated  with  this  process  are  nonlinear,  and  hence  the  terminology.  In  this  paper 
we  consider  the  large  deviation  properties  of  the  particle  system  as  the  number  of  particles  tends 
to  infinity.  Thus  the  deviations  we  study  are  those  of  the  empirical  measure  of  the  prelimit  process 
from  the  distribution  of  the  nonlinear  diffusion.  In  the  paper  we  develop  an  approach  which  is 
very  different  from  the  one  taken  in  previous  works.  Unlike  classical  techniques,  our  proofs  do  not 
involve  any  time  or  space  discretization  of  the  system  and  no  exponential  probability  estimates 
are  invoked.  The  main  ingredients  in  the  proof  are  weak  convergence  methods  for  functional  oc¬ 
cupation  measures  and  certain  variational  representation  formulas.  Our  proofs  cover  models  with 
degenerate  noise  and  allow  for  interaction  in  both  drift  and  diffusion  terms.  In  fact  the  techniques 
are  applicable  to  a  wide  range  of  model  settings  and  an  example  of  stochastic  delay  equations  is 
considered  to  illustrate  the  possibilities.  The  starting  point  of  our  analysis  is  a  variational  repre¬ 
sentation  for  moments  of  nonnegative  functionals  of  a  Brownian  motion.  Using  this  representation, 
the  proof  of  the  large  deviation  principle  reduces  to  the  study  of  asymptotic  properties  of  certain 
controlled  versions  of  the  original  process.  The  key  step  in  the  proof  is  to  characterize  the  weak 
limits  of  the  control  and  controlled  process  as  the  large  deviation  parameter  tends  to  its  limit  and 
under  the  same  scaling  that  applies  to  the  original  process.  More  precisely,  one  needs  to  charac¬ 
terize  the  limit  of  the  empirical  measure  of  a  large  collection  of  controlled  and  weakly  interacting 
processes.  In  the  absence  of  control  this  characterization  problem  reduces  to  an  LLN  analysis  of 
the  original  particle  system,  which  has  been  studied  extensively.  Our  main  tools  for  the  study  of 
the  controlled  analogue  are  functional  occupation  measure  methods.  Indeed,  these  methods  have 
been  found  to  be  quite  useful  for  the  study  of  averaging  problems,  but  where  the  average  is  with 
respect  to  a  time  variable.  In  the  problem  studied  here  the  measure  valued  processes  of  interest  are 
obtained  using  averaging  over  particles  rather  than  the  time  variable.  Variational  representations 
for  Brownian  motions  and  Poisson  random  measures  have  proved  to  be  extremely  useful  for  the 
study  of  small  noise  large  deviation  problems  and  many  recent  papers  have  applied  these  results  to 
a  variety  of  infinite  dimensional  small  noise  systems.  We  expect  the  current  work  to  be  similarly 
a  starting  point  for  the  study,  using  variational  representations,  of  a  rather  different  collection  of 
large  deviation  problems,  namely  asymptotics  of  a  large  number  of  interacting  particles. 


(II)  On  Near  Optimal  Trajectories  of  Games  Associated  with  the  infinity-Laplacian 

[3] .  Consider  the  equation 


— 2/looU  =  h  in  G, 

< 

u  =  g  on  8G, 


(1) 


where,  for  an  integer  m  >  2,  G  C  R,n  is  a  bounded  C2  domain,  and  g  £  C(dG,R)  and  the  functions 
h  £  C(G,  M  \  {0})  are  given.  The  infinity-Laplacian  is  defined  as 
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provided  Df  ^  0,  where  for  a  C 2  function  /  we  denote  by  Df  the  gradient  and  by  D2f  the  Hessian 
matrix.  This  paper  is  motivated  by  recent  work  of  Peres  et.  al.  [35],  where  a  discrete  time  random 
turn  game,  referred  to  as  Tug-of-War,  is  developed  in  relation  to  (1).  This  game,  parameterized 
by  e  >  0,  has  the  property  that  the  vanishing-e  limit  of  the  value  function  uniquely  solves  (1)  in 
the  viscosity  sense.  The  stochastic  differential  equation  (SDE) 

dXt  =  2 p(Xt)dWt  +  2  q(Xt)dt,  (2) 

where  ^ 

P=p||.  q  =  jp^(D2uDu-A0OuDu),  (3) 

is  suggested  in  [35]  as  the  game’s  dynamics  in  the  vanishing-e  limit.  The  relation  is  rigorously 
established  in  examples,  but  only  heuristically  justified  in  general.  Recently,  in  [2],  a  two-player 
zero-sum  stochastic  differential  game  (SDG)  is  considered,  for  which  the  value  function  uniquely 
solves  (1)  in  the  viscosity  sense.  The  dynamics  of  the  state  process  are  given  as 

Xt  =  x  +  f  (As  —  Bs)dWs  +  f  ( Cs  +  DS)(AS  +  Bs)d-s ,  t  €  [0,  oo),  (4) 

Jo  Jo 

and  (A,  C )  and  ( B ,  D)  are  control  processes,  chosen  by  the  two  players,  taking  values  in  5m_1  x 
[0,  oo)  where  <Sm_1  is  the  unit  sphere  in  Mm.  Defined  in  the  Elliott-Kalton  sense,  the  SDG  of  [2] 
is  formulated  in  such  a  way  that  one  of  the  players  selects  a  strategy,  and  then  the  other  selects  a 
control  process.  The  payoff  functional  of  interest  is  an  exit  time  cost  of  the  form  E[  fj  h(Xs)ds  + 
g(XT)\,  where  r  =  inf{t  :  Xt  (f  G\  (with  an  appropriate  convention  regarding  r  =  oo). 

The  goal  of  this  paper  is  to  show  that,  with  appropriate  conditions,  (2)  can  be  rigorously 
interpreted  as  the  optimal  dynamics  of  the  SDG.  We  will  assume  in  this  paper  that  the  equation 
possesses  a  classical  solution  u  i.e.,  C 2  with  non- vanishing  gradient.  Under  this  assumption  we 
specify,  for  each  8  >  0,  a  h-optimal  strategy  /3s ,  and  a  control  process  Ys  that  is  h-optimal  for  play 
against  /3s,  in  terms  of  first  and  second  derivatives  of  u.  We  then  identify  the  limit  law,  as  5  — >•  0, 
of  the  state  process  under  (/3s,  Ys),  as  the  solution  X  to  the  SDE  (2),  stopped  when  X  hits  the 
boundary  dG. 

The  construction  of  near  optimal  strategy- control  pairs,  that  is  of  interest  in  its  own  right,  is 
based  on  an  interpretation  of  (1)  as  the  following  Bellman-Isaacs  type  equation 

sup  inf  {  —  -  (a  —  b)'  (D2u)(a  —  b)  —  (c  +  d)(a  +  b)  ■  Du\  =  h. 

H=M>o  M=hc>o  f  2  J 

In  this  form  there  is  a  natural  way  to  construct  strategy  and  control,  by  associating  the  supremum 
and  inhmum  with  the  two  players.  The  variables  a,  b,  c  and  d  selected  by  the  players  dictate  the 
coefficients  of  the  game’s  state  process,  and,  we  prove,  the  coefficients  converge  to  those  of  equation 
(2)  in  the  limit  as  the  supremum  and  infimum  are  achieved.  This  convergence  is  then  lifted  to  the 
convergence  of  the  underlying  state  process  in  (4)  to  the  diffusion  (2). 


(Ill)  Variational  Representations  for  Continuous  Time  Processes  [18].  In  this  paper  we 
prove  a  variational  representation  for  positive  measurable  functionals  of  a  Poisson  random  measure 
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and  an  infinite  dimensional  Brownian  motion.  These  processes  provide  the  driving  noises  for  a 
wide  range  of  important  process  models  in  continuous  time,  and  thus  we  also  obtain  variational 
representations  for  these  processes  when  a  strong  solution  exists.  The  representations  have  a  number 
of  uses,  the  most  important  being  to  prove  large  deviation  estimates. 

The  theory  of  large  deviations  is  by  now  well  understood  in  many  settings,  but  there  remain 
some  situations  where  the  topic  is  not  as  well  developed.  These  are  often  settings  where  technical 
issues  challenge  standard  approaches,  and  the  problem  of  finding  nearly  optimal  or  even  reasonably 
weak  sufficient  conditions  is  hindered  as  much  by  technique  of  proof  as  any  other  issue. 

Variational  representations  of  the  sort  developed  in  this  paper  have  been  shown  to  be  partic¬ 
ularly  useful,  when  combined  with  weak  convergence  methods,  for  analyzing  such  systems.  For 
example,  Brownian  motion  representations  have  been  used  by  many  authors  (see  references  in  [18]) 
in  the  large  deviation  analysis  of  solutions  to  SPDEs  in  the  small  noise  limit.  The  usefulness  of  the 
representations  is  in  part  due  to  the  fact  that  they  avoid  certain  discretization  and/or  approxima¬ 
tion  arguments,  which  can  be  cumbersome  for  complex  systems.  Another  reason  is  that  exponential 
tightness,  a  property  that  is  often  required  by  other  approaches  and  which  often  leads  to  artifi¬ 
cial  conditions,  is  replaced  by  ordinary  tightness  for  controlled  processes  with  uniformly  bounded 
control  costs.  (Although  exponential  tightness  can  a  posteriori  be  obtained  as  consequence  of  the 
large  deviation  principle  (LDP)  and  properties  of  the  rate  function).  What  is  required  for  the  weak 
convergence  approach,  beyond  the  variational  representations,  is  that  basic  qualitative  properties 
(existence,  uniqueness  and  law  of  large  number  limits)  can  be  demonstrated  for  certain  controlled 
versions  of  the  original  process. 

Previous  work  on  variational  representations  has  focused  on  either  discrete  time  processes, 
functionals  of  finite  dimensional  Brownian  motion,  or  various  formulations  of  infinite  dimensional 
Brownian  motion.  An  important  class  of  processes  that  were  not  covered  are  continuous  time 
Markov  processes  with  jumps,  e.g.,  Levy  processes.  In  this  paper  we  eliminate  this  gap,  and  in 
fact  give  variational  representations  for  functionals  of  a  fairly  general  Poisson  random  measure 
(PRM)  plus  an  independent  infinite  dimensional  Brownian  motion  (BM),  thereby  covering  many 
continuous  time  models. 

In  [38]  Zhang  has  also  proved  a  variational  representation  for  functionals  of  a  PRM.  The  repre¬ 
sentation  in  [38]  is  given  in  terms  of  certain  predictable  transformations  on  the  canonical  Poisson 
space.  Existence  of  such  transformations  relies  on  solvability  of  certain  nonlinear  partial  differen¬ 
tial  equations  from  the  theory  of  mass  transportation.  This  imposes  restrictive  conditions  on  the 
intensity  measure  (e.g.,  absolute  continuity  with  respect  to  Lebesgue  measure)  of  the  PRM,  and 
even  the  very  elementary  setting  of  a  standard  Poisson  process  is  not  covered.  Additionally,  use  of 
such  a  representation  for  proving  large  deviation  results  for  general  continuous  time  models  with 
jumps  appears  to  be  unclear. 

In  contrast,  we  impose  very  mild  assumptions  on  the  intensity  measure  (namely,  it  is  a  u-finite 
measure  on  a  locally  compact  space),  and  establish  a  representation,  that  is  given  in  terms  of  a  fixed 
PRM  defined  on  an  augmented  space.  A  key  question  in  formulating  the  representation  for  PRM 
is  “ what  form  of  controlled  PRM  is  natural  for  purposes  of  representation?’’’’  In  the  Brownian  case 
there  is  little  room  for  discussion,  since  control  by  shifting  the  mean  is  obviously  very  appealing.  In 
[38]  the  control  moves  the  atoms  of  the  Poisson  random  measure  through  a  rather  complex  nonlinear 
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transformation.  The  fact  that  atoms  are  neither  created  nor  destroyed  is  partly  responsible  for  the 
fact  that  the  representation  does  not  cover  the  standard  Poisson  process.  In  the  representation 
obtained  in  our  work  the  control  process  enters  as  a  censoring/thinning  function  in  a  very  concrete 
fashion,  which  in  turn  allows  for  elementary  weak  convergence  arguments  in  proofs  of  large  deviation 
results. 

As  an  application  of  the  representation,  we  establish  a  general  large  deviation  principle  (LDP) 
for  functionals  of  a  PRM  and  an  infinite  dimensional  BM.  A  similar  LDP  for  functionals  of  an 
infinite  dimensional  BM  [16]  has  been  used  in  recent  years  by  numerous  authors  to  study  small 
noise  asymptotics  for  a  variety  of  infinite  dimensional  stochastic  dynamical  models  (see  references 
in  [18]).  The  LDP  obtained  in  the  current  paper  is  expected  to  be  similarly  useful  in  the  study 
of  infinite  dimensional  stochastic  models  with  jumps  (e.g.,  SPDEs  with  jumps).  In  our  work  we 
illustrate  the  use  of  the  LDP  in  via  a  simple  finite  dimensional  jump-diffusion  model.  The  goal  is 
to  simply  show  how  the  approach  can  be  used  and  no  attempt  is  made  to  obtain  the  best  possible 
conditions. 


(IV)  Ergodic  Rate  Control  Problem  for  Single  Class  Queueing  Networks  [19],  We  study 
Jackson  networks  with  state  dependent  and  dynamically  controlled  arrival  and  service  rates.  The 
network  consists  of  K  service  stations,  each  of  which  has  an  associated  infinite  capacity  buffer. 
Arrivals  of  jobs  can  be  from  outside  the  system  and/or  from  internal  routing.  Upon  completion  of 
service  at  a  station,  the  customer  is  routed  to  one  of  the  other  service  stations  (or  exits  the  system) 
according  to  a  probabilistic  routing  matrix.  Network  is  assumed  to  be  in  heavy  traffic  in  an 
appropriate  sense.  Roughly  speaking,  one  considers  a  sequence  of  networks  with  identical  topology, 
parametrized  by  n  £  N.  Instantaneous  arrival  and  service  rates  in  the  n-th  network  are  queue 
length  dependent  and  are  of  order  0(n).  Additionally,  a  system  manager  can  exercise  service  rate 
controls  of  order  0(n1//2).  The  heavy  traffic  assumption  says  that  traffic  intensity  at  each  station 
is  of  the  order  1  —  0(1/ in 1//2).  In  an  uncontrolled  setting,  diffusion  approximations  of  suitably 
scaled  queue  length  processes  for  such  networks  have  been  studied  in  [37].  The  limit  stochastic 
process  is  a  reflected  diffusion  in  the  nonnegative  orthant.  Dependence  of  arrival  and  service  rates 
on  queue  length  processes  leads  to  drift  and  diffusion  coefficients  for  the  limit  model  that  are  state 
dependent.  Existence  and  uniqueness  of  solutions  to  such  reflected  stochastic  differential  equations 
(SDE)  follows  from  the  classical  theory  and  well  understood  regularity  properties  of  the  Skorohod 
map. 

For  the  controlled  setting  considered  here,  a  formal  analysis  along  similar  lines  leads  to  controlled 
reflected  diffusions  with  control  entering  linearly  in  the  drift  coefficient.  Goal  of  this  work  is  to 
use  such  a  formal  diffusion  approximation  in  order  to  obtain  provably  asymptotically  optimal  rate 
control  policies  for  the  underlying  physical  networks.  We  are  concerned  with  an  average  long  term 
cost  per  unit  time  criterion.  Cost  function  is  a  sum  of  two  terms;  the  first  term  measures  the 
cost  for  holding  customers  in  the  buffer,  while  the  second  term  is  the  cost  for  exercising  control. 
The  holding  cost  is  allowed  to  have  a  polynomial  growth  (as  a  function  of  queue  lengths)  while 
the  control  cost  is  assumed  to  be  linear.  Since  the  cost  criterion  involves  an  infinite  time  horizon, 
additional  stability  conditions  are  needed  to  ensure  the  finiteness  of  the  cost.  In  our  work  these 
conditions  are  manifested  in  the  definition  of  the  class  of  admissible  controls.  The  definition  in 
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particular  ensures  that  the  controls  are  “uniformly  stabilizing”,  viz.  all  polynomial  moments  of 
the  queue  length  process  are  bounded  uniformly  in  time,  the  scaling  parameter  and  the  choice  of 
admissible  control  sequence.  This  stability  property  is  a  key  ingredient  in  our  proofs. 

Our  main  result  relates  the  value  function  (i.e.  the  optimum  value  of  the  cost)  of  the  queueing 
rate  control  problem  with  the  value  function  associated  with  an  ergodic.  cost  problem  for  the  formal 
controlled  diffusion  limit.  Cost  function  for  the  diffusion  model  is  analogous  to  that  for  the  physical 
networks.  It  is  shown  that,  under  certain  conditions,  the  value  function  for  the  n-th  queueing 
network  converges  to  that  for  the  diffusion  model,  as  n  ->  oo.  The  theorem  also  shows  that  using 
an  appropriately  chosen  e-optimal  feedback  control  for  the  diffusion  model,  one  can  construct  an 
asymptotically  e-optimal  rate  control  policy  for  the  physical  queueing  network.  Thus  the  theorem 
describes  the  precise  sense  in  which  the  diffusion  model,  which  is  originally  justified  only  formally, 
is  a  rigorous  asymptotic  approximation  for  the  queueing  model.  Rate  control  problems  with  an 
ergodic  cost  criterion  for  single  class  open  queueing  networks  in  heavy  traffic  have  been  studied  in 
several  papers.  Most  works  concern  a  one  dimensional  problem  (i.e.  a  single  server  queue)  and 
provide  explicit  expressions/recipes  for  asymptotically  optimal  policies.  The  paper  [32]  studies  a 
general  K  dimensional  Jackson  type  network  with  finite  buffers.  In  this  setting,  due  to  compactness 
of  the  state  space,  all  stability  issues  become  trivial.  Furthermore,  convergence  of  value  functions 
is  established  under  the  additional  assumption  that  e-optimal  continuous  feedback  controls  exist 
for  the  limit  diffusion  model-  but  no  details  on  when  such  an  assumption  holds  are  provided. 

One  of  the  main  steps  in  the  proof  of  our  main  result  is  establishing  existence  of  an  e-optimal 
continuous  feedback  control  for  the  limit  diffusion  model.  Existence  of  optimal  feedback  controls 
for  the  class  of  ergodic  diffusion  control  problems  studied  here  has  been  established  in  [10].  Our 
proof  starts  from  such  an  optimal  control  b*  and  constructs  a  sequence  {bn}  of  continuous  feedback 
controls  in  a  manner  such  that  an  appropriately  chosen  measure  of  the  set  {bn  ^  b*}  converges  to  0 
as  n  —>  oo.  We  show  that,  as  long  as  initial  distributions  converge,  the  solution  Xn  of  the  reflected 
SDE  that  uses  bn  as  the  feedback  control  converges  weakly  to  X  which  solves  the  equation  with  bn 
replaced  with  b*.  Proof  of  this  result  relies  on  certain  representations  and  estimates  for  transition 
probability  densities  of  reflected  Brownian  motions  that  are  of  independent  interest.  Once  weak 
convergence  of  Xn  to  X  is  established,  existence  of  an  e-optimal  continuous  feedback  control  is  an 
immediate  consequence  of  the  linearity  of  the  control  cost.  Next,  an  e-optimal  continuous  feedback 
control  b£ ,  obtained  from  above  result  is  used  to  define  a  rate  control  policy  for  the  n-th  network, 
for  each  n  G  N.  The  associated  costs  are  shown  to  converge  to  the  cost  associated  with  b£  in  the 
diffusion  control  problem.  As  a  consequence  of  this  result  one  obtains  that  the  value  function  for 
the  limit  diffusion  model  is  an  upper  bound  for  any  limit  point  of  the  sequence  of  value  functions 
for  the  controlled  queueing  networks.  We  then  establish  the  reverse  inequality.  Combining  these 
bounds  we  have  the  main  result  of  the  paper.  Proofs  of  these  bounds  use  functional  occupation 
measure  methods  developed  in  [32],  The  main  idea  is  to  represent  the  cost  associated  with  the  n-th 
network  in  terms  of  integrals  with  respect  to  occupation  measures  associated  with  various  stochastic 
processes  that  constitute  the  dynamical  description  of  the  network.  Using  stability  estimates  of 
this  paper  one  can  establish  tightness  of  these  occupation  measures  and  then  classical  martingale 
characterization  methods  can  be  applied  to  identify  the  limit  points  in  an  appropriate  manner. 

Stability  estimates  obtained  in  this  work  are  also  useful  for  the  study  of  the  uncontrolled  set¬ 
ting.  We  show  that,  under  appropriate  conditions,  stationary  distributions  of  scaled  queue  length 
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processes  (which  are  strong  Markov  processes  when  there  are  no  controls)  converge  to  the  unique 
stationary  distribution  of  the  limit  reflecting  diffusion  model.  For  the  setting  where  the  arrival  and 
service  rates  are  constant,  such  a  result  has  been  proved  in  [21].  To  our  knowledge  the  current 
paper  is  the  first  one  to  treat  a  setting  with  state  dependent  rates. 


(V)  Multiscale  Diffusion  Approximations  for  Stochastic  Networks  in  Heavy  Traffic[23]. 

Jackson  networks  have  been  very  well  studied  and  extensively  used  for  modeling  and  analysis 
in  a  variety  of  disciplines.  The  elegant  distributional  theory  and  asymptotic  properties  of  Jackson 
networks  break  down  when  one  attempts  to  incorporate  some  more  realistic  features  of  specific 
application  settings  into  such  a  model.  For  example,  when  the  distributions  of  the  primitives 
are  relaxed  to  be  more  general  than  i.i.d.  exponential,  or  the  rates  of  interarrivals  and  services  are 
allowed  to  depend  on  the  state  of  the  system,  the  tractability  of  such  a  generalized  Jackson  network 
fails  and  thus  one  seeks  suitable  approximate  models.  One  class  of  such  approximations  are  diffusion 
models  that  can  be  rigorously  justified  when  networks  are  operating  in  the  heavy  traffic  regime, 
i.e.,  when  the  network  capacity  is  roughly  balanced  with  network  load.  Attractiveness  of  such 
approximations  primarily  lies  in  the  fact  that,  analogous  to  the  central  limit  theory,  the  limit  model 
is  described  only  using  a  few  important  parameters  of  the  underlying  networks  and  the  complex 
distributional  properties  of  the  primitives  are  averaged  out.  The  first  general  result  in  the  study  of 
such  diffusion  approximations  is  due  to  Reiman  [36]  who  considered  the  case  where  the  arrivals  and 
services  are  mutually  independent  renewal  processes  with  square  integrable  summands.  Queueing 
networks  in  which  rates  of  arrival  and  service  processes  depend  on  the  current  state  of  network 
arise  in  many  application  areas.  In  particular,  Yamada[37]  has  shown  that,  under  appropriate 
heavy  traffic  conditions,  suitably  scaled  queue  length  processes  for  such  networks  converge  weakly 
to  a  reflected  diffusion  process  with  drift  and  diffusion  coefficients  that  depend  on  the  state  of  the 
process. 

In  models  considered  in  works  of  Reiman  and  Yamada,  the  underlying  topology  of  the  network 
is  the  same  as  that  of  a  Jackson  network.  In  particular,  there  is  a  fixed  probability  routing  matrix  P 
which  governs  the  routing  of  jobs  in  the  networks.  In  this  work,  we  are  interested  in  settings  where 
the  routing  structure  and  arrival/service  rates  may  change  over  time,  according  to  an  extraneous 
finite  state  Markov  process.  This  Markov  process  can  be  interpreted  as  the  random  environment  in 
which  the  system  is  operating.  We  begin  by  considering  a  situation  where  the  system  performance 
is  analyzed  on  time  scales  which  are  much  larger  than  typical  interarrival/service  times  of  jobs, 
which  in  turn  are  significantly  larger  than  typical  time  intervals  at  which  the  state  of  the  random 
environment  changes.  Such  multi-scale  models  are  motivated  by  the  study  of  large  computer 
networks  where  one  is  interested  in  the  behavior  of  modeling  traffic  of  files  with  moderate  size  over 
a  long  period  of  time,  for  a  small  subset  of  nodes  in  the  system.  Denote  by  £  the  collection  of  all 
nodes  in  the  network  and  £q  C  £  the  subset  of  nodes  of  interest.  Here,  the  size  of  £$  is  much  smaller 
than  that  of  £.  One  is  interested  in  building  a  model  for  traffic  between  nodes  in  £$  without  taking 
a  very  precise  account  of  the  interactions  of  such  nodes  with  those  in  £  \  £$ .  A  node  e  £  £q  nray 
receive  files  from  a  large  number  of  nodes,  most  of  which  one  does  not  want  to  explicitly  account 
for  in  the  reduced  model.  The  point  of  view  taken  here  is  to  model  the  effect  of  nodes  in  £  \£q 
at  a  node  e  £  £q  by  a  rapidly  varying  channel  capacity  (at  e),  which  is  modulated  by  a  Markov 
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process.  If  a  large  number  of  nodes  in  £\£q  are  connected  to  e,  one  expects  that  the  rate  at  which 
the  channel  capacity  changes  is  much  greater  than  the  transmission  rate  of  a  typical  file  through 
e.  Rapid  changes  in  channel  capacity  lead  to  variations  in  processing  rates  and  available  routing 
options  for  files  accepted  at  nodes  in  £q.  Thus  we  propose  a  traffic  model  for  nodes  in  £q  in  term 
of  a  Jackson  type  network  where  the  arrival/service  rates  and  routing  probability  matrices  vary 
randomly  over  time  according  to  a  finite  state  Markov  process. 

Goal  of  this  work  is  to  study  diffusion  approximations  for  such  multiscale  models.  We  show 
that,  under  appropriate  conditions,  the  state  of  the  system  can  be  well  approximated  by  that  of  a 
reflected  diffusion  process  with  drift  and  diffusion  coefficients  depend  on  the  equilibrium  measure 
of  the  Markov  process  governing  the  random  environment.  A  crucial  requirement  in  typical  heavy 
traffic  approximation  results  is  the  traffic  balance  condition  of  the  form  A  =  [I  —  P]a.  In  the  multi¬ 
scale  setting  considered  here  all  these  quantities  in  the  above  equation,  i.e.,  P,  A,  a,  vary  randomly 
over  time  according  to  the  modulating  Markov  process.  We  find  that  if  transition  times  of  the 
Markov  process  are  suitably  fast,  then  the  traffic  balance  condition  can  be  relaxed  to  an  average 
balance  condition  where  the  average  is  taken  with  respect  to  the  equilibrium  measure  of  the  Markov 
process.  Making  this  statement  mathematically  precise  is  one  of  the  key  contributions  of  this  work. 

In  contrast  to  the  scaling  regime  described  above  is  the  situation  where  the  random  environment 
changes  very  slowly  relative  to  the  rates  of  interarrival  and  service  times.  In  such  a  regime,  changes 
in  the  environment  are  revealed  only  when  the  system  is  viewed  over  long  time  intervals.  Our 
second  result  in  this  work  is  to  give  a  precise  mathematical  formulation  and  establish  a  suitable 
diffusion  approximation  for  networks  in  such  a  scaling  regime.  For  simplicity  we  restrict  ourselves 
to  a  setting  where  the  random  environment  affects  only  the  arrival  and  service  rates  in  the  system 
while  the  routing  probabilities  stay  the  same  (governed  by  a  fixed  non-random  matrix  P).  We  show 
that,  with  a  suitable  scaling,  the  state  process  for  the  network  approaches  a  reflected  diffusion  with 
coefficients  that  are  modulated  by  a  Markov  process  that  is  independent  of  the  driving  Brownian 
motion. 

The  two  scaling  regimes  considered  above  correspond  to  very  different  behaviors  of  the  back¬ 
ground  environment  process.  In  the  first  setting  the  environment  changes  at  a  much  faster  rate 
than  the  typical  arrival/serive  rates  in  the  system,  while  in  the  second  one  the  reverse  is  true.  More 
generally,  one  can  consider  a  setting  where  background  variation  of  both  types  exists.  Namely, 
there  are  two  independent  Markov  processes  governing  the  randomly  varying  environment,  the  first 
corresponds  to  fast  changes  while  the  second  captures  slow  changes.  We  also  establish  an  appro¬ 
priate  diffusion  approximation  result  for  such  multi-scale  queueing  networks.  This  work  is  part  of 
the  dissertation  work  of  Ms.  Xin  Liu. 

(VI)  Action  Time  Sharing  Policies  for  Ergodic  Control  of  Markov  Chains  [27].  Markov 
Decision  processes  are  used  extensively  as  the  simplest  models  that  involve  both  stochastic  behavior 
and  control.  A  common  measure  of  performance  is  the  long-time  average  (or  ergodic)  criterion. 
Given  all  relevant  parameters,  a  typical  goal  is  to  find  a  simple  (e.g.  feedback,  or  deterministic 
stationary)  policy  that  achieves  the  optimal  value. 

The  goal  of  adaptive  control  is  to  obtain  an  optimal  policy,  when  some  relevant  information  con¬ 
cerning  the  behavior  of  the  system  is  missing.  The  relevant  information  needs  to  be  obtained  while 
controls  are  chosen  at  each  step.  The  classical  approach  is  to  design  an  algorithm  which  collects 


information,  while  at  the  same  time  choosing  controls,  in  such  a  way  that  sufficient  information  is 
collected  for  making  good  control  decisions,  in  the  sense  that  the  chosen  controls  “approach  opti¬ 
mality  over  time.”  Existing  results  include  general  solutions  for  the  case  of  countable  state  space, 
and  specify  an  estimation  and  a  control  scheme.  A  different  approach  to  this  issue,  including  PAC 
criteria,  can  be  found  in  the  large  literature  on  Reinforcement  learning. 

We  are  concerned  with  a  more  elementary  question,  namely:  What  are  the  basic  controlled 
objects  that  determine  the  cost?  Since  the  objective  function  is  defined  as  a  Cesaro  limit,  we 
can  expect  that  a  similar  Cesaro  definition  of  the  choice  of  controls  would  suffice  to  determine 
the  cost.  For  the  case  of  countable  state  and  action  spaces  one  fundamental  result  says  that  the 
control  decisions  can  deviate  from  those  dictated  by  the  Markov  policy  q,  and  still  produce  the 
same  long  term  average  cost,  as  long  as  the  conditional  frequencies  converge  to  the  correct  values. 
This  flexibility  is  useful  in  many  estimation  and  sampling  applications.  In  the  current  work  we  are 
concerned  with  a  setting  where  the  state  and  action  spaces  are  not  (necessarily)  countable.  Our 
main  objective  is  to  formulate  an  appropriate  definition  for  an  Action-Time  Sharing  (ATS)  policy 
for  a  given  Markov  control  which,  similar  to  the  countable  case,  on  the  one  hand  leads  to  long  term 
costs  that  are  identical  to  those  for  the  corresponding  Markov  control,  while  on  the  other  hand  allows 
for  flexible  implementation  well  suited  for  various  estimation  and  adaptive  control  goals.  We  show 
that,  under  suitable  stability,  irreducibility  and  Feller  continuity  conditions,  occupation  measures 
for  state  and  action  sequences,  under  an  ATS  policy  (suitably  defined),  converge  a.s.  to  the  same 
(deterministic)  measure  as  under  the  corresponding  Markov  control.  Such  a  result  in  particular 
shows  that  long  term  costs  for  a  broad  family  of  one  stage  cost  functions,  under  the  two  control 
policies,  coincide.  ATS  policies  can  be  used  to  develop  a  variety  of  variance  reduction  schemes  for 
ergodic  control  problems.  Additionally,  ATS  policies  provide  much  flexibility  for  sampling  (namely 
using  controls  without  regards  to  the  ensuing  cost),  for  example  for  the  purpose  of  collecting 
information.  This  could  be  information  which  is  related  to  the  main  optimization  objective,  but 
could  also  be  other  information  which  is  of  interest.  We  show  that  how  ATS  policies  introduced  in 
this  work  can  be  used  for  consistent  estimation  of  unknown  model  parameters  and  also  for  adaptive 
control  problems.  This  paper  is  part  of  the  dissertation  work  of  Ms.  Xin  Liu. 

(VII)  Exit  Time  and  Invariant  Measure  Asymptotics  for  Small  Noise  Constrained 
Diffusions  [6] .  Diffusions  in  polyhedral  domains  arise  commonly  as  approximate  models  for 
stochastic  processing  networks  in  heavy  traffic.  In  this  work  we  consider  a  family  of  such  constrained 
diffusions  with  a  small  parameter  (denoted  as  e)  multiplying  the  diffusion  coefficient.  Goal  of  the 
work  is  the  study  of  asymptotic  properties  of  invariant  measures  and  exit  times  from  suitable 
domains,  as  e  — >  0.  The  classical  reference  for  small  noise  asymptotics  of  diffusions  in  is  the 
book  of  Freidlin  and  Wentzell  (1998).  The  basic  object  of  study  in  this  fundamental  body  of  work 
is  a  collection  of  diffusion  processes  {Xe}e>o,  given  as 

dXe(t )  =  b(X£(t.))dt  +  Ea(Xe(t))dW(t),  Xe(0)  =  x,  (5) 

where  W  is  a  k  dimensional  standard  Brownian  motion  and  b,  a  are  suitable  coefficients.  The 
stochastic  process  Xe  =  {Xs (t)}o<t<T>  for  each  T  >  0  can  be  regarded  as  a  Ct  =  C'([0,  T]  :  Mfc) 
(space  of  continuous  functions  from  [0,  T]  to  with  the  uniform  topology)  valued  random  variable 
and  under  suitable  conditions  on  b,  a  one  can  show  that,  as  e  — >■  0,  Xs  converges  in  probability  to 
£  which  is  the  unique  solution  of  the  ordinary  differential  equation  (ODE) 

i  =  ho,  m  =  *■  (e) 
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One  of  the  basic  results  in  the  field  says  that  for  each  T  >  0,  as  e  — >•  0,  X£  satisfies  a  large 
deviation  principle  (LDP)  in  Ct,  uniformly  in  the  initial  condition  x  in  any  compact  set  K,  with  an 
appropriate  rate  function  It  :  Ct  — »  [0,  oo].  This  result  is  a  starting  point  for  the  study  of  numerous 
asymptotic  questions  for  such  small  noise  diffusions.  In  particular,  when  the  underlying  diffusions 
have  suitable  stability  properties,  the  above  LDP  plays  a  central  role  in  the  study  of  asymptotic 
properties  of  invariant  measures  and  exit  times  from  domains.  The  asymptotics  are  governed  by  the 
“quasi-potential”  function  V  which  is  determined  from  the  collection  of  rate  functions  {It  :  T  >  0}. 

Freidlin-Wentzell  theory  has  been  extended  and  refined  in  many  different  directions.  One  no¬ 
table  work  is  Day(1982)  which  studies  asymptotics  of  solutions  of  Dirichlet  problems  associated 
with  diffusions  given  by  (5).  To  signify  the  dependence  on  the  initial  condition,  denote  the  solution 
of  (5)  by  X£.  Let  B  be  a  bounded  domain  in  and  K  be  an  arbitrary  compact  subset  of  B. 
Under  the  assumption  that  all  solutions  of  the  ODE  (6),  with  x  =  £(0)  €  B,  converge  without 
leaving  B,  to  a  single  linearly  asymptotically  stable  critical  point,  Day  shows  that  with  suitable 
conditions  on  the  coefficients,  for  all  bounded  measurable  / 

sup  | IE  (f(X£(r£)))  -  IE  (/ {Xy(Ty)))  | 

x,y£K 

converges  to  0  at  an  exponential  rate.  Here,  rj  =  inf{f  :  X£{t)  €  Bc}.  This  property,  usually 
referred  to  as  “exponential  leveling”,  says  that  although  the  exit  time  of  the  process  from  the 
domain  approaches  oo,  as  e  — >•  0,  the  moments  of  functionals  of  exit  location,  corresponding  to 
distinct  initial  conditions,  coalesce  asymptotically,  at  an  exponential  rate.  The  key  ingredient  in 
the  proof  is  the  gradient  estimate 

sup  |V«£(x)|  <  c£-1//2,  (7) 

x$LK 

where  u£  is  the  solution  of  the  Dirichlet  problem  on  B  associated  with  the  diffusion  (5)  with 
boundary  data  /. 

The  goal  of  the  current  work  is  to  develop  the  Freidlin-Wentzell  small  noise  theory  for  a  family 
of  constrained  diffusions  in  polyhedral  cones.  Let  G  C  be  convex  polyhedral  cone  in  with 
the  vertex  at  origin  given  as  the  intersection  of  half  spaces  Gi,i  =  1,2,  ...,7V.  Let  nt  be  the  unit 
vector  associated  with  Gi  via  the  relation 

Gi  =  {.x  G  Mfc  :  (. x,rii )  >  0}. 

We  will  denote  the  set  {x  G  dG  :  (x,rii)  =  0}  by  Ft.  With  each  face  Ft  we  associate  a  unit  vector 
di  such  that  {di,  rii)  >  0.  This  vector  defines  the  direction  of  constraint  associated  with  the  face  lq. 
Roughly  speaking  such  a  process  evolves  infinitesimally  as  a  diffusion  in  and  is  instantaneously 
pushed  back  using  the  oblique  reflection  direction  di  upon  reaching  the  face  Ft.  Formally,  such  a 
process,  denoted  once  more  as  X£,  can  be  represented  as  a  solution  of  a  stochastic  integral  equation 
of  the  form 

X£x{t)  =  r(x  +  f  b(X£(s))ds  +  e  a(X£ (s))dW {s)  \  (t),  (8) 

where  T  is  the  Skorohod  map  taking  trajectories  with  values  in  to  those  with  values  in  G , 
consistent  with  the  constraint  vectors  {di,i  =  l,---iV}.  Under  certain  regularity  assumptions  on 
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the  Skorohod  map  and  the  usual  Lipschitz  conditions  on  the  coefficients  b  and  a,  our  first  result 
establishes  a  (locally  uniform)  LDP  for  X£,  in  C([().  T]  :  G )  for  each  T  >  0.  This  result  is  the 
starting  point  for  all  exit  time  and  invariant  measure  estimates  obtained  in  this  work. 


Stability  properties  of  constrained  diffusions  in  polyhedral  domains  have  been  studied  in  several 
works.  Let 


C  =  {-£  aidi  :  cti  >  0;  i  €  {1, ...,  k}}. 


i=  1 


The  paper  [5]  shows  that  under  regularity  of  the  Skorohod  map,  uniform  non-degeneracy  of  er  and 
Lipschitz  coefficients,  if  for  some  <5  >  0 


b(x)  €  C(5)  ={»6C  :  dist(u,  dC)  >  5}  for  all  x  G  G , 


(9) 


then  the  constrained  diffusion  X£  is  positive  recurrent  and  consequently  admits  a  unique  invariant 
probability  measure  fi£.  In  the  current  work  we  study  asymptotic  properties  of  n£ ,  as  s  — >  0. 

We  also  consider  asymptotic  properties  of  exit  times  from  a  bounded  domain  B  C  G  that 
contains  the  origin.  One  important  feature  in  this  analysis  is  that  the  stability  condition  (9),  in 
general,  does  not  ensure  that  the  trajectories  of  the  associated  deterministic  dynamical  system 


£  =  r(.x+  [  b(£(s))ds)  (10) 

Jo 

with  initial  condition  in  B  will  stay  within  the  domain  at  all  times.  However,  a  weaker  stability 
property  holds,  namely,  one  can  find  domains  Bq  C  B  such  that  all  trajectories  of  (10)  starting  in 
Bq  stay  within  B  at  all  times. 

A  significant  part  of  this  work  is  devoted  to  the  proof  of  the  exponential  leveling  property 
for  constrained  diffusions.  We  recall  that  the  key  ingredient  in  the  proof  of  such  a  result  for 
diffusions  in  is  the  gradient  estimate  (7)  for  solutions  of  the  associated  Dirichlet  problem.  For 
diffusions  in  domains  with  corners  and  with  oblique  reflection  fields  that  change  discontinuously, 
there  are  no  regularity  (eg.  C 1  solutions)  results  known  for  the  associated  partial  differential 
equations(PDE).  Our  proof  of  the  exponential  leveling  property  is  purely  probabilistic  and  bypasses 
all  PDE  estimates.  The  main  step  in  the  proof  is  the  construction  of  certain  (uniform  in  e) 
Lyapunov  functions  which  are  then  used  to  construct  a  coupling  of  the  processes  X£,  X£  with 
explicit  uniform  estimates  on  exponential  moments  of  time  to  coupling.  The  key  ingredient  in  this 
coupling  construction  is  a  minorization  condition  on  transition  densities  of  reflected  diffusions.  We 
give  some  examples  where  such  a  minorization  property  holds.  Obtaining  general  conditions  under 
which  this  estimate  on  transition  densities  of  reflected  diffusions  holds  is  a  challenging  open  problem. 
The  minorization  property  allows  for  the  construction  of  a  pseudo-atom,  for  each  fixed  e  >  0,  using 
split  chain  ideas  of  Athreya-Ney-Nummelin.  Coupling  based  on  pseudo-atom  constructions  have 
been  used  by  many  authors  for  the  study  of  a  broad  range  of  stability  and  control  problems,  however 
the  current  paper  appears  to  be  the  first  to  bring  these  powerful  techniques  to  bear  for  the  study 
of  exit  time  asymptotics  of  small  noise  Markov  processes. 

As  a  second  consequence  of  our  coupling  constructions  we  show  that  difference  of  moments  of 
an  exit  time  functional  with  a  sub-logarithmic  growth  corresponding  to  distinct  initial  conditions  in 
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Bo  is  asymptotically  bounded.  Note  that  for  typical  unbounded  functionals  the  associated  moments 
will  approach  oo,  as  e  — >  0  thus  the  result  provides  a  rather  non-trivial  “leveling  estimate”.  The 
third  important  consequence  of  our  approach  says  that  as  initial  conditions  approach  0  at  rate  e2 
the  corresponding  moments  of  exit  time  functionals  with  sub-logarithmic  growth  asymptotically 
coalesce  at  an  exponential  rate.  To  the  best  of  our  knowledge,  analogous  estimates  are  not  known 
even  for  the  setting  of  small  noise  diffusions  (with  suitable  stability  properties)  in  Mfc. 

These  estimates  are  interesting  for  one  dimensional  models  as  well.  Let  X  be  a  one  dimensional 
reflected  Brownian  motion,  starting  from  x  €  [0,1),  with  drift  b  €  (— oo,0)  and  variance  a2e2  € 
(0,  oo),  given  on  some  probability  space  (Q,T,P£X).  I.e.,  P£x  a.e., 

X(t )  =  x  +  bt  +  eoB{t )  —  inf  {(x  +  bs  +  eaB(s))  AO},  t  >  0 

0<s<t 

where  B  is  a  standard  Brownian  motion.  Let  r  =  infjf  :  X(t)  =  1}.  Let  V’  :  — >•  1R+  be  a 

measurable  map  and  let  <£>£(x)  =  E xip(r).  Our  results  for  this  one  dimensional  setting  says  that  if 
if)  has  a  sub- logarithmic  growth  then  for  some  c,  5  €  (0,  oo),  as  e  — >  0, 

e5/£\(p£(ce2)  -  </?£(0)|  ->•  0.  (11) 

If  ip{t)  =  t,  t  >  0,  then  ipe  is  the  unique  solution  to  the  boundary  value  problem 

e?°2<p"  +  b<p'£  =  -1,  ¥>' (0)  =  0,  <pe{  1)  =  0, 

2  2  —  b  — bx  ..  ^  _ 

whose  solution  is  given  as  <p£(x)  =  ^-[e^2^2  —  ePXI]  +  ^{l  —  x) .  It  is  easily  verified  that  if  xe  =  ce2, 
for  some  c  >  0,  then  \ip£(x£)  —  <^£(0)|  — >  0  at  rate  e2,  as  e  — >  0.  In  particular,  the  decay  is  not 
exponential.  Furthermore,  for  x  ^  y  in  [0,1),  \ipe(x)  —  tpe(y) I  — >  oo  as  e  — >  0.  For  general  i\) 
a  similar  ODE  analysis  is  less  tractable  and  thus  the  coupling  techniques  developed  here  give  a 
powerful  approach  (especially  in  higher  dimensions)  to  the  study  of  such  asymptotic  estimates. 

(VIII)  Controlled  Stochastic  Networks  in  Heavy  Traffic:  Convergence  of  Value  Func¬ 
tions  [29].  As  an  approximation  to  control  problems  for  critically- loaded  stochastic  networks, 
Harrison  (1988)  has  formulated  a  stochastic  control  problem  in  which  the  state  process  is  driven 
by  a  multi-dimensional  Brownian  motion  along  with  an  additive  control  that  satisfies  certain  fea¬ 
sibility  and  non-negativity  constraints.  This  control  problem  that  is  usually  referred  to  as  the 
Brownian  Control  Problem  (BCP)  has  been  one  of  the  key  developments  in  the  heavy  traffic  theory 
of  controlled  stochastic  processing  networks  (SPN).  BCPs  can  be  regarded  as  formal  scaling  limits 
for  a  broad  range  of  scheduling  and  sequencing  control  problems  for  multiclass  queuing  networks. 
Finding  optimal  (or  even  near-optimal)  control  policies  for  such  networks  -  which  may  have  quite 
general  non- Markovian  primitives,  multiple  server  capabilities  and  rather  complex  routing  geom¬ 
etry  -  is  prohibitive.  In  that  regard  BCPs  that  provide  significantly  more  tractable  approximate 
models  are  very  useful.  In  this  diffusion  approximation  approach  to  policy  synthesis,  one  first  finds 
an  optimal  (or  near-optimal)  control  for  the  BCP  which  is  then  suitably  interpreted  to  construct 
a  scheduling  policy  for  the  underlying  physical  network.  In  recent  years  there  have  been  many 
works  that  consider  specific  network  models  for  which  the  associated  BCP  is  explicitly  solvable  (i.e. 
an  optimal  control  process  can  be  written  as  a  known  function  of  the  driving  Brownian  motions) 
and,  by  suitably  adapting  the  solution  to  the  underlying  network,  construct  control  policies  that 
are  asymptotically  (in  the  heavy  traffic  limit)  optimal.  Although  now  there  are  several  papers 
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which  establish  a  rigorous  connection  between  a  network  control  problem  and  its  associated  BCP 
by  exploiting  the  explicit  form  of  the  solution  of  the  latter,  a  systematic  theory  which  justifies  the 
use  of  BCPs  as  approximate  models  has  been  missing.  In  a  recent  work  [28]  it  was  shown  that  for 
a  large  family  of  Unitary  Networks,  with  general  interarrival  and  service  times,  probabilistic  rout¬ 
ing,  and  an  infinite  horizon  discounted  linear  holding  cost,  the  cost  associated  with  any  admissible 
control  policy  for  the  network  is  asymptotically,  in  the  heavy  traffic  limit,  bounded  below  by  the 
value  function  of  the  BCP.  This  inequality,  which  provides  a  useful  bound  on  the  best  achievable 
asymptotic  performance  for  an  admissible  control  policy  (for  this  large  family  of  models),  was  a  key 
step  in  developing  a  rigorous  general  theory  relating  BCPs  with  SPN  in  heavy  traffic.  The  current 
paper  is  devoted  to  the  proof  of  the  reverse  inequality.  The  network  model  is  required  to  satisfy 
assumptions  made  in  [28].  In  addition  we  impose  a  non-degeneracy  condition,  a  condition  on  the 
underlying  renewal  processes  regarding  probabilities  of  deviations  from  the  mean  and  regularity  of 
a  certain  Skorohod  map.  Under  these  assumptions  we  prove  that  the  value  function  of  the  BCP 
is  bounded  below  by  the  heavy  traffic  limit  (limsup)  of  the  value  functions  of  the  network  control 
problem.  Combining  this  with  the  result  obtained  in  [28]  we  obtain  the  main  result  of  the  paper. 
This  theorem  says  that,  under  broad  conditions,  the  value  function  of  the  network  control  prob¬ 
lem  converges  to  that  of  the  BCP.  This  result  provides,  under  general  conditions,  a  rigorous  basis 
for  regarding  BCPs  as  approximate  models  for  critically  loaded  stochastic  networks.  Conditions 
imposed  in  this  paper  allow  for  a  wide  range  of  open  multi-class  queuing  networks,  parallel  server 
networks  and  job  shop  type  models.  We  note  that  our  approach  does  not  require  the  BCP  to 
be  explicitly  solvable  and  the  result  covers  many  settings  where  explicit  solutions  are  unavailable. 
Previous  works  noted  earlier,  that  treat  the  setting  of  explicitly  solvable  BCP,  do  much  more  than 
establish  convergence  of  value  functions.  In  particular,  these  works  give  an  explicit  implementable 
control  policy  for  the  underlying  network  that  is  asymptotically  optimal  in  the  heavy  traffic  limit. 
In  the  generality  treated  in  the  current  work,  giving  explicit  recipes  (eg.  threshold  type  policies)  is 
unfeasible,  however  the  policy  sequence  that  we  construct,  suggests  a  general  approach  for  building 
near  asymptotically  optimal  policies  for  the  network  given  a  near  optimal  control  for  the  BCP. 
Obtaining  near  optimal  controls  for  the  BCP  in  general  requires  numerical  approaches,  study  of 
which  will  be  a  topic  for  future  work. 


(IX)  Stability  of  Constrained  Markov  Modulated  Diffusions  [30]. 

A  family  of  constrained  diffusions  in  a  random  environment  is  considered.  Constraint  set  is  a 
polyhedral  cone  and  coefficients  of  the  diffusion  are  governed  by,  in  addition  to  the  system  state, 
a  finite  state  Markov  process  that  is  independent  of  the  driving  noise.  Such  models  arise  as  limit 
objects  in  the  heavy  traffic  analysis  of  stochastic  processing  networks  (SPN)  with  Markov  mod¬ 
ulated  arrival  and  processing  rates.  We  give  sufficient  conditions  (which  in  particular  includes  a 
requirement  on  the  regularity  of  the  underlying  Skorohod  map)  for  positive  recurrence  and  geo¬ 
metric  ergodicity.  When  the  coefficients  only  depend  on  the  modulating  Markov  process  (i.e.  they 
are  independent  of  the  system  state),  a  complete  characterization  for  stability  and  transience  is 
provided.  The  case,  where  the  pathwise  Skorohod  problem  is  not  well-posed  but  the  underlying 
reflection  matrix  is  completely-5,  is  treated  as  well.  As  consequences  of  geometric  ergodicity  vari¬ 
ous  results,  such  as  exponential  integrability  of  invariant  measures  and  CLT  for  fluctuations  of  long 
time  averages  of  process  functionals  about  their  stationary  values,  are  obtained.  Conditions  for  sta- 
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bility  are  formulated  in  terms  of  the  averaged  drift,  where  the  average  is  taken  with  respect  to  the 
stationary  distribution  of  the  modulating  Markov  process.  Finally,  steady  state  distributions  of  the 
underlying  SPN  are  considered  and  it  is  shown  that,  under  suitable  conditions,  such  distributions 
converge  to  the  unique  stationary  distribution  of  the  constrained  random  environment  diffusion. 


(X)  Discrete  Time  Markovian  Agents  Interacting  Through  a  Potential  [15].  A  discrete 
time  stochastic  model  for  a  multiagent  system  given  in  terms  of  a  large  collection  of  interacting 
Markov  chains  is  studied.  The  evolution  of  the  interacting  particles  is  described  through  a  time 
inhomogeneous  transition  probability  kernel  that  depends  on  the  ‘gradient’  of  the  potential  field. 
The  particles,  in  turn,  dynamically  modify  the  potential  field  through  their  cumulative  input.  In¬ 
teracting  Markov  processes  of  the  above  form  have  been  suggested  as  models  for  active  biological 
transport  in  response  to  external  stimulus  such  as  a  chemical  gradient.  One  of  the  basic  mathemat¬ 
ical  challenges  is  to  develop  a  general  theory  of  stability  for  such  interacting  Markovian  systems 
and  for  the  corresponding  nonlinear  Markov  processes  that  arise  in  the  large  agent  limit.  Such  a 
theory  would  be  key  to  a  mathematical  understanding  of  the  interactive  structure  formation  that 
results  from  the  complex  feedback  between  the  agents  and  the  potential  held.  It  will  also  be  a 
crucial  ingredient  in  developing  simulation  schemes  that  are  faithful  to  the  underlying  model  over 
long  periods  of  time.  The  goal  of  this  work  is  to  study  qualitative  properties  of  the  above  stochastic 
system  as  the  number  of  particles  ( N  )  and  the  time  parameter  (n)  approach  infinity.  In  this  regard 
asymptotic  properties  of  a  deterministic  nonlinear  dynamical  system,  that  arises  in  the  propagation 
of  chaos  limit  of  the  stochastic  model,  play  a  key  role.  We  show  that  under  suitable  conditions 
this  dynamical  system  has  a  unique  fixed  point.  This  result  allows  us  to  study  stability  proper¬ 
ties  of  the  underlying  stochastic  model.  We  show  that  as  JV  ->  oo,  the  stochastic  system  is  well 
approximated  by  the  dynamical  system,  uniformly  over  time.  As  a  consequence,  for  an  arbitrarily 
initialized  system,  as  N  — >•  oo  and  n  — >•  oo,  the  potential  held  and  the  empirical  measure  of  the 
interacting  particles  are  shown  to  converge  to  the  unique  fixed  point  of  the  dynamical  system.  In 
general,  simulation  of  such  interacting  Markovian  systems  is  a  computationally  daunting  task.  We 
propose  a  particle  based  approximation  for  the  dynamic  potential  held  which  allows  for  a  numer¬ 
ically  tractable  simulation  scheme.  It  is  shown  that  this  simulation  scheme  well  approximates  the 
true  physical  system,  uniformly  over  an  inhnite  time  horizon. 


(XI)  Bohman- Frieze  processes  at  criticality  and  emergence  of  the  giant  component  [7]. 

The  percolation  phase  transition  and  the  mechanism  of  the  emergence  of  the  giant  component 
through  the  critical  scaling  window  for  random  graph  models,  has  been  a  topic  of  great  interest 
in  many  different  communities  ranging  from  statistical  physics,  combinatorics,  computer  science, 
social  networks  and  probability  theory.  The  last  few  years  have  witnessed  an  explosion  of  models 
which  couple  random  aggregation  rules,  that  specify  how  one  adds  edges  to  existing  configurations, 
and  choice,  wherein  one  selects  from  a  limited  set  of  edges  at  random  to  use  in  the  configuration. 
These  models  exhibit  fascinating  new  phenomenon,  ranging  from  delaying  or  speeding  up  the  emer¬ 
gence  of  the  giant  component,  to  explosive  percolation,  where  the  diameter  of  the  scaling  window  is 
several  orders  of  magnitude  smaller  than  that  for  standard  random  graph  models.  While  an  intense 
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study  of  such  models  has  ensued,  understanding  the  actual  emergence  of  the  giant  component  and 
merging  dynamics  in  the  critical  scaling  window  has  remained  impenetrable  to  a  rigorous  analysis. 
In  this  work  we  take  an  important  step  in  the  analysis  of  such  models  by  studying  one  of  the  stan¬ 
dard  examples  of  such  processes,  namely  the  Bohman-Frieze  model,  and  provide  first  results  on  the 
asymptotic  dynamics,  through  the  critical  scaling  window,  that  lead  to  the  emergence  of  the  giant 
component  for  such  models.  We  identify  the  scaling  window  and  show  that  through  this  window, 
the  component  sizes  properly  rescaled  converge  to  the  standard  multiplicative  coalescent.  Proofs 
hinge  on  a  careful  analysis  of  certain  infinite-type  branching  processes  with  types  taking  values  in 
the  space  of  RCLL  paths,  and  stochastic  analytic  techniques  to  estimate  susceptibility  functions  of 
the  components  all  the  way  through  the  scaling  window  where  these  functions  explode.  Previous 
approaches  for  analyzing  random  graphs  at  criticality  have  relied  largely  on  classical  breadth-first 
search  techniques  that  exploit  asymptotic  connections  with  Brownian  excursions.  For  dynamic 
random  graph  models  evolving  via  general  Markovian  rules,  such  approaches  fail  and  we  develop 
a  quite  different  set  of  tools  that  can  potentially  be  used  for  the  study  of  critical  dynamics  for  all 
bounded  size  rules. 


(XII)  A  Numerical  Scheme  for  Invariant  Distributions  of  Constrained  Diffusions  [12]. 

Reflected  diffusions  in  polyhedral  domains  are  commonly  used  as  approximate  models  for 
stochastic  processing  networks  in  heavy  traffic.  Stationary  distributions  of  such  models  give  use¬ 
ful  information  on  the  steady  state  performance  of  the  corresponding  stochastic  networks  and 
thus  it  is  important  to  develop  reliable  and  efficient  algorithms  for  numerical  computation  of  such 
distributions.  In  this  work  we  propose  and  analyze  a  Monte-  Carlo  scheme  based  on  an  Euler 
type  discretization  of  the  reflected  stochastic  dierential  equation  using  a  single  sequence  of  time 
discretization  steps  which  decrease  to  zero  as  time  approaches  infinity.  Appropriately  weighted 
empirical  measures  constructed  from  the  simulated  discretized  reflected  diffusion  are  proposed  as 
approximations  for  the  invariant  probability  measure  of  the  true  diffusion  model.  Almost  sure 
consistency  results  are  established  that  in  particular  show  that  weighted  averages  of  polynomially 
growing  continuous  functionals  evaluated  on  the  discretized  simulated  system  converge  a.s.  to  the 
corresponding  integrals  with  respect  to  the  invariant  measure.  Proofs  rely  on  constructing  suitable 
Lyapunov  functions  for  tightness  and  uniform  integrability  and  characterizing  almost  sure  limit 
points  through  an  extension  of  Echeverrias  criteria  for  reflected  diffusions.  Regularity  properties  of 
the  underlying  Skorohod  problems  play  a  key  role  in  the  proofs.  Rates  of  convergence  for  suitable 
families  of  test  functions  are  also  obtained.  A  key  advantage  of  Monte-Carlo  methods  is  the  ease 
of  implementation,  particularly  for  high  dimensional  problems.  A  numerical  example  of  a  eight 
dimensional  Skorohod  problem  is  presented  to  illustrate  the  applicability  of  the  approach. 


(XIII)  Near  Critical  Catalyst  Reactant  Branching  Processes  with  Controlled  Immi¬ 
gration  [31].  Near  critical  catalyst-reactant  branching  processes  with  controlled  immigration  are 
studied.  The  reactant  population  evolves  according  to  a  branching  process  whose  branching  rate 
is  proportional  to  the  total  mass  of  the  catalyst.  The  bulk  catalyst  evolution  is  that  of  a  classical 
continuous  time  branching  process;  in  addition  there  is  a  specific  form  of  immigration.  Immigration 
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takes  place  exactly  when  the  catalyst  population  falls  below  a  certain  threshold,  in  which  case  the 
population  is  instantaneously  replenished  to  the  threshold.  Such  models  are  motivated  by  prob¬ 
lems  in  chemical  kinetics  where  one  wants  to  keep  the  level  of  a  catalyst  above  a  certain  threshold 
in  order  to  maintain  a  desired  level  of  reaction  activity.  A  diffusion  limit  theorem  for  the  scaled 
processes  is  presented,  in  which  the  catalyst  limit  is  described  through  a  reflected  diffusion,  while 
the  reactant  limit  is  a  diffusion  with  coefficients  that  are  functions  of  both  the  reactant  and  the 
catalyst.  Stochastic  averaging  principles  under  fast  catalyst  dynamics  are  established.  In  the  case 
where  the  catalyst  evolves  “much  faster”  than  the  reactant,  a  scaling  limit,  in  which  the  reactant  is 
described  through  a  one  dimensional  SDE  with  coefficients  depending  on  the  invariant  distribution 
of  the  reffected  diffusion,  is  obtained.  Proofs  rely  on  constrained  martingale  problem  characteriza¬ 
tions,  Lyapunov  function  constructions,  moment  estimates  that  are  uniform  in  time  and  the  scaling 
parameter,  and  occupation  measure  techniques. 


(XIV)  Large  Deviations  for  Stochastic  Partial  Differential  Equations  Driven  by  a  Pois¬ 
son  Random  Measure  [11].  Stochastic  partial  differential  equations  driven  by  Poisson  random 
measures  (PRM)  have  been  proposed  as  models  for  many  different  physical  systems,  where  they 
are  viewed  as  a  refinement  of  a  corresponding  noiseless  partial  differential  equations  (PDE).  A 
systematic  framework  for  the  study  of  probabilities  of  deviations  of  the  stochastic  PDE  from  the 
deterministic  PDE  is  through  the  theory  of  large  deviations.  The  goal  of  this  work  is  to  develop 
the  large  deviation  theory  for  small  Poisson  noise  perturbations  of  a  general  class  of  deterministic 
infinite  dimensional  models.  Although  the  analogous  questions  for  finite  dimensional  systems  have 
been  well  studied,  there  are  currently  no  general  results  in  the  infinite  dimensional  setting.  This  is 
in  part  due  to  the  fact  that  in  this  setting  solutions  may  have  little  spatial  regularity,  and  thus  clas¬ 
sical  approximation  methods  for  large  deviation  analysis  become  intractable.  The  approach  taken 
here,  which  is  based  on  a  variational  representation  for  nonnegative  functionals  of  general  PRM, 
reduces  the  proof  of  the  large  deviation  principle  to  establishing  basic  qualitative  properties  for 
controlled  analogues  of  the  underlying  stochastic  system.  As  an  illustration  of  the  general  theory, 
we  consider  a  particular  system  that  models  the  spread  of  a  pollutant  in  a  waterway. 

(XV)  Confidence  regions  for  stochastic  variational  inequalities  [34].  The  sample  average 
approximation  (SAA)  method  is  a  basic  approach  for  solving  stochastic  variational  inequalities. 
It  is  well  known  that  the  SAA  solutions  provide  good  point  estimators  for  the  true  solution  to  a 
stochastic  variational  inequality.  It  is  of  fundamental  interest  to  use  such  point  estimators  along 
with  the  central  limit  theory  to  develop  confidence  regions  of  prescribed  level  of  significance  for 
the  true  solution.  However,  the  non-smooth  nature  of  variational  inequalities  makes  it  difficult 
to  construct  the  confidence  regions  by  following  standard  procedures.  This  paper  overcomes  such 
difficulty  and  proposes  a  method  to  build  asymptotically  exact  confidence  regions  for  the  true 
solution  that  are  computable  from  the  SAA  solutions.  We  justify  this  method  theoretically  by 
establishing  a  precise  limit  theorem,  apply  it  to  complementarity  problems,  and  test  it  with  a 
linear  complementarity  problem. 

(XVI)  Infinite  Dimensional  Forward- Backward  Stochastic  Differential  Equations  and 
the  KPZ  Equation  [1].  Kardar-Parisi-Zhang  (KPZ)  equation  is  a  quasilinear  stochastic  partial 
differential  equation(SPDE)  driven  by  a  space-time  white  noise.  In  recent  years  there  have  been 
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several  works  directed  towards  giving  a  rigorous  meaning  to  a  solution  of  this  equation.  Bertini, 
Cancrini  and  Giacomin  have  proposed  a  notion  of  a  solution  through  a  limiting  procedure  and 
a  certain  renormalization  of  the  nonlinearity.  In  this  work  we  study  connections  between  the 
KPZ  equation  and  certain  infinite  dimensional  forward-backward  stochastic  differential  equations. 
Forward-backward  equations  with  a  finite  dimensional  noise  have  been  studied  extensively,  mainly 
motivated  by  problems  in  mathematical  finance.  Equations  considered  here  differ  from  the  classical 
works  in  that,  in  addition  to  having  an  infinite  dimensional  driving  noise,  the  associated  SPDE 
involves  a  non-Lipschitz  (namely  a  quadratic)  function  of  the  gradient.  Existence  and  uniqueness 
of  solutions  of  such  infinite  dimensional  forward-backward  equations  is  established  and  the  terminal 
values  of  the  solutions  are  then  used  to  give  a  new  probabilistic  representation  for  the  solution  of 
the  KPZ  equation. 

(XVII)  Admission  Control  for  Multidimensional  Workload  with  Heavy  Tails  and  Frac¬ 
tional  Ornstein-Uhlenbeck  Process.  [26].  The  infinite  source  Poisson  arrival  model  with 
heavy-tailed  workload  distributions  has  attracted  much  attention,  especially  in  the  modeling  of 
data  packet  traffic  in  communication  networks.  In  particular,  it  is  well  known  that  under  suitable 
assumptions  on  the  source  arrival  rate,  the  centered  and  scaled  cumulative  workload  process  for 
the  underlying  processing  system  can  be  approximated  by  fractional  Brownian  motion.  In  many 
applications  one  is  interested  in  the  stabilization  of  the  work  inflow  to  the  system  by  modify¬ 
ing  the  net  input  rate,  using  an  appropriate  admission  control  policy.  In  this  work  we  study  a 
natural  family  of  admission  control  policies  which  keep  the  associated  scaled  cumulative  workload 
asymptotically  close  to  a  pre-specified  linear  trajectory,  uniformly  over  time.  Under  such  admission 
control  policies  and  with  natural  assumptions  on  arrival  distributions,  suitably  scaled  and  centered 
cumulative  workload  processes  are  shown  to  converge  weakly  in  the  path  space  to  the  solution  of 
a  d-dimensional  stochastic  differential  equation  (SDE)  driven  by  a  Gaussian  process.  It  is  shown 
that  the  admission  control  policy  achieves  moment  stabilization  in  that  the  second  moment  of  the 
solution  to  the  SDE  (averaged  over  the  d-stations)  is  bounded  uniformly  for  all  times.  In  one  special 
case  of  control  policies,  as  time  approaches  infinity,  we  obtain  a  fractional  version  of  a  stationary 
Ornstein-Uhlenbeck  process  that  is  driven  by  fractional  Brownian  motion  with  Hurst  parameter 
H  >  1/2. 

(XVIII)  Dynamic  Scheduling  for  Markov  Modulated  Single-server  Multiclass  Queue¬ 
ing  Systems  in  Heavy  Traffic.  [20].  This  paper  studies  a  scheduling  control  problem  for  a 
single-server  multiclass  queueing  network  in  heavy  traffic,  operating  in  a  changing  environment. 
The  changing  environment  is  modeled  as  a  finite  state  Markov  process  that  modulates  the  arrival 
and  service  rates  in  the  system.  Various  cases  are  considered:  fast  changing  environment,  fixed 
environment  and  slow  changing  environment.  In  each  of  the  cases,  using  weak  convergence  analysis, 
in  particular  functional  limit  theorems  for  renewal  processes  and  ergodic  Markov  processes,  it  is 
shown  that  an  appropriate  “averaged”  version  of  the  classical  c/i  -policy  (the  priority  policy  that  fa¬ 
vors  classes  with  higher  values  of  the  product  of  holding  cost  c  and  service  rate  /i)  is  asymptotically 
optimal  for  an  infinite  horizon  discounted  cost  criterion. 

(XIX)  Bounded-size  rules:  The  barely  subcritical  regime.  [8]  Bounded-size  rules  are 
dynamic  random  graph  processes  which  incorporate  limited  choice  along  with  randomness  in  the 
evolution  of  the  system.  One  starts  with  the  empty  graph  and  at  each  stage  two  edges  are  chosen 
uniformly  at  random.  One  of  the  two  edges  is  then  placed  into  the  system  according  to  a  decision 
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rule  based  on  the  sizes  of  the  components  containing  the  four  vertices.  For  bounded-size  rules,  all 
components  of  size  greater  than  some  fixed  K  >  1  are  accorded  the  same  treatment.  Writing  BS(f) 
for  the  state  of  the  system  with  nt/2  edges,  Spencer  and  Wormald  proved  that  for  such  rules,  there 
exists  a  critical  time  tc  such  that  when  t  <tc  the  size  of  the  largest  component  is  of  order  log  n  while 
for  t  >  tc,  the  size  of  the  largest  component  is  of  order  n.  In  this  work  we  obtain  upper  bounds  (that 
hold  with  high  probability)  of  order  n 27  log4  n,  on  the  size  of  the  largest  component,  at  time  instants 
tn  =  tc—n~ 7,  where  7  £  (0, 1/4).  This  result  for  the  barely  subcritical  regime  forms  a  key  ingredient 
in  the  study  undertaken  in  [9],  of  the  asymptotic  dynamic  behavior  of  the  process  describing  the 
vector  of  component  sizes  and  associated  complexity  of  the  components  for  such  random  graph 
models  in  the  critical  scaling  window.  The  proof  uses  a  coupling  of  BSR  processes  with  a  certain 
family  of  inhomogeneous  random  graphs  with  vertices  in  the  type  space  M+  x  D([ 0,  00)  :  No)  where 
D([0, 00)  :  No)  is  the  Skorohod  D-space  of  functions  that  are  right  continuous  and  have  left  limits 
equipped  with  the  usual  Skorohod  topology.  The  coupling  construction  also  gives  an  alternative 
characterization  (than  the  usual  explosion  time  of  the  susceptibility  function)  of  the  critical  time 
tc  for  the  emergence  of  the  giant  component  in  terms  of  the  operator  norm  of  integral  operators  on 
certain  L 2  spaces. 

(XX)  The  augmented  multiplicative  coalescent  and  critical  dynamic  random  graph 
models.  [9]  Random  graph  models  with  limited  choice  have  been  studied  extensively  with  the  goal 
of  understanding  the  mechanism  of  the  emergence  of  the  giant  component.  One  of  the  standard 
models  are  the  Achlioptas  random  graph  processes  on  a  fixed  set  of  n  vertices.  Here  at  each  step, 
one  chooses  two  edges  uniformly  at  random  and  then  decides  which  one  to  add  to  the  existing 
configuration  according  to  some  criterion.  An  important  class  of  such  rules  are  the  bounded-size 
rules  where  for  a  fixed  K  >  1,  all  components  of  size  greater  than  K  are  treated  equally.  While 
a  great  deal  of  work  has  gone  into  analyzing  the  subcritical  and  supercritical  regimes,  the  nature 
of  the  critical  scaling  window,  the  size  and  complexity  (deviation  from  trees)  of  the  components 
in  the  critical  regime  and  nature  of  the  merging  dynamics  has  not  been  well  understood.  In  this 
work  we  study  such  questions  for  general  bounded-size  rules.  Our  first  main  contribution  is  the 
construction  of  an  extension  of  Aldous’s  standard  multiplicative  coalescent  process  which  describes 
the  asymptotic  evolution  of  the  vector  of  sizes  and  surplus  of  all  components.  We  show  that  this 
process,  referred  to  as  the  standard  augmented  multiplicative  coalescent  (AMC)  is  ‘nearly’  Feller 
with  a  suitable  topology  on  the  state  space.  Our  second  main  result  proves  the  convergence  of 
suitably  scaled  component  size  and  surplus  vector,  for  any  bounded-size  rule,  to  the  standard  AMC. 
The  key  ingredients  here  are  a  precise  analysis  of  the  asymptotic  behavior  of  various  susceptibility 
functions  near  criticality  and  certain  bounds  from  [8],  on  the  size  of  the  largest  component  in  the 
barely  subcritical  regime. 

(XXI)  Central  Limit  Results  for  Jump-Diffusions  with  Mean  Field  Interaction  and  a 
Common  Factor  [22]  A  system  of  N  weakly  interacting  particles  whose  dynamics  is  given  in  terms 
of  jump-diffusions  with  a  common  factor  is  considered.  The  common  factor  is  described  through 
another  jump-diffusion  and  the  coefficients  of  the  evolution  equation  for  each  particle  depend,  in 
addition  to  its  own  state  value,  on  the  empirical  measure  of  the  states  of  the  N  particles  and  the 
common  factor.  Systems  with  a  common  factor  arise  in  many  different  areas.  In  Mathematical 
Finance,  they  have  been  used  to  model  correlations  between  default  probabilities  of  multiple  firms. 
In  neuroscience  modeling  these  arise  as  systematic  noise  in  the  external  current  input  to  a  neuronal 
ensemble.  For  particle  approximation  schemes  for  stochastic  partial  differential  equations  (SPDE), 
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the  common  factor  corresponds  to  the  underlying  driving  noise  in  the  SPDE.  The  goal  of  this 
work  is  to  study  a  general  family  of  weakly  interacting  jump-diffusions  with  a  common  factor.  Our 
main  objective  is  to  establish  a  suitable  Central  Limit  Theorem(CLT).  A  key  point  here  is  that 
due  to  the  presence  of  the  common  factor,  the  limit  of  particle  empirical  measures  will  in  general 
be  a  random  measure.  This  in  particular  means  that  the  centering  in  the  fluctuation  theorem 
will  typically  be  random  as  well  and  one  expects  the  limit  law  for  such  fluctuations  to  be  not 
Gaussian  but  rather  a  ‘Gaussian  mixture’.  Our  main  result  provides  a  CLT  under  quite  general 
conditions  .  The  summands  in  this  CLT  can  be  quite  general  functionals  of  the  trajectories  of  the 
particles  with  suitable  integrability  properties.  The  key  idea  is  to  first  consider  a  closely  related 
collection  of  N  stochastic  processes  that,  conditionally  on  a  common  factor,  are  independent  and 
identically  distributed.  By  introducing  a  suitable  Radon-Nikodym  derivative  one  can  evaluate  the 
expectations  associated  with  a  perturbed  form  of  the  original  scaled  and  centered  sum  in  terms  of 
the  conditionally  i.i.d.  collection.  The  asymptotics  of  the  latter  quantity  are  easier  to  analyze  using, 
in  particular,  the  classical  limit  theorems  for  symmetric  statistics.  The  perturbation  arises  due  to 
the  fact  that  in  the  original  system  the  evolution  of  the  common  factor  jump-diffusion  depends  on 
the  empirical  measure  of  the  states  of  the  IV-particles  whereas  in  the  conditionally  i.i.d.  construction 
the  common  factor  evolution  is  determined  by  the  large  particle  limit  of  the  empirical  measures. 
Estimating  the  error  introduced  by  this  perturbation  is  one  of  the  key  technical  challenges  in  the 
proof.  An  application  to  models  in  Mathematical  Finance  of  self-excited  correlated  defaults  is 
described. 

(XXII)  On  the  multi-dimensional  skew  Brownian  motion  [4],  We  provide  a  new,  concise 
proof  of  weak  existence  and  uniqueness  of  solutions  to  the  stochastic  differential  equation  for  the 
multidimensional  skew  Brownian  motion.  We  also  present  an  application  to  Brownian  particles 
with  skew-elastic  collisions. 

Let  E  denote  the  hyperplane  {x  €  W1,  :  x\  =  0}  in  Rd,  d  >  1,  and  let  a  vector  field  b  :  E  — >• 
be  given  on  it.  Consider  the  stochastic  differential  equation  (SDE),  for  a  process  X  taking  values 
in  Rrf,  of  the  form 

X(t)  =  x  +  W(t)  +  [  b(X(s))dL(s),  t>  0,  (12) 

Jo 

where  x  €  W  is  a  standard  d-dimensional  Brownian  motion  and  L  is  the  local  time  of  X  at  E, 
and  b  is  bounded  and  Lipschitz,  and  satisfies  bi(x)  €  [—1, 1]  for  all  x  €  E.  This  paper  provides  a  new 
proof  of  weak  existence  and  uniqueness  of  solutions  to  (12).  A  more  general  equation  that  allows 
for  a  bounded,  measurable  drift  coefficient  is  also  treated.  These  results  are  known  by  the  work  of 
Takanobu(1986,1987).  Our  purpose  here  is  to  provide  a  much  shorter  proof.  Takanobu  studies  in 
a  sequence  of  two  papers  an  equation  of  the  form  (12)  with  general  drift  and  diffusion  coefficients. 
The  first  paper  treats  existence  of  weak  solutions  and  the  second  proves  uniqueness.  Existence  is 
shown  by  constructing  a  suitable  approximating  diffusion  with  a  non-singular  drift  while  uniqueness 
relies  on  rather  involved  arguments  from  Brownian  excursion  theory.  While  our  results  are  a  special 
case  of  those  established  by  Takanobu,  the  proof  (of  both  existence  and  uniqueness)  provided  here 
is  much  shorter  and  more  elementary.  Existence  of  solutions  is  established  by  constructing  a  ‘skew 
random  walk’  and  showing  that  in  diffusion  scaling  it  converges  in  distribution  to  a  weak  solution 
of  the  equation.  The  proof  of  weak  uniqueness  provided  here  is  inspired  by  the  technique  used 
in  Weinryb(1984)  in  a  one-dimensional  setting  with  time-varying  skewness.  As  an  application  we 
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consider  a  model  introduced  by  Fernholz  et  al.  (2013)  for  the  dynamics  of  a  pair  of  1-dinrensional 
Brownian  particles  X\  and  X2  that  exhibit  various  possible  types  of  interaction  when  they  collide. 
The  equations  involve  the  local  time  at  zero  of  the  relative  position,  and  the  types  of  interaction 
are  determined  by  the  coefficients  in  front  of  the  local  time  terms.  In  contrast  to  Fernholz  et  al. 
our  work  allows  for  state-dependent  local  time  coefficients.  This  allows  one  to  model  variability  in 
the  type  of  collision,  where  the  type  is  determined  by  the  collision  position. 

(XXIII)  Moderate  Deviation  Principles  for  Stochastic  Differential  Equations  with 
Jumps  [17].  Large  deviation  principles  for  small  noise  diffusion  equations  have  been  extensively 
studied  in  the  literature.  Since  the  original  work  of  Freidlin  and  Wentzell(1970),  model  assumptions 
have  been  significantly  relaxed  and  many  extensions  have  been  studied  in  both  finite-dimensional 
and  infinite- dimensional  settings. 

The  goal  of  this  work  is  to  study  moderate  deviation  problems  for  stochastic  dynamical  systems. 
In  such  a  study  one  is  concerned  with  probabilities  of  deviations  of  a  smaller  order  than  in  large 
deviation  theory.  Consider  for  example  an  independent  and  identically  distributed  (iid)  sequence 
{T)}j>i  of  Revalued  zero  mean  random  variables  with  common  probability  law  p.  A  large  deviation 
principle  (LDP)  for  Sn  =  Ya= 1  will  formally  say  that  for  c  >  0 

W(\Sn\  >  nc )  «  exp{-ninf{/(y)  :  |y|  >  c}}, 

where  for  y  €  Rd,  I(y)  =  supagRd{(a,  y)  —  log  exp(a,  y)p(dy)}.  Now  let  {a„}  be  a  positive 
sequence  such  that  an  f  00  and  n_1//2an  — >  0  as  n  — X  00  (e.g.  an  =  n1/4).  Then  a  moderate 
deviation  principle  (MDP)  for  Sn  will  say  that 

P(|Sn|  >  n1/2anc)  «  exp{— a2  inf{/°(y)  :  |y|  >  c}}, 

where  I°(y)  =  \  (y,  and  E  =  Cov(T).  Thus  the  moderate  deviation  principle  gives  estimates 

on  probabilities  of  deviations  of  order  n1//2an  which  is  of  lower  order  than  n  and  with  a  rate  function 
that  is  a  quadratic  form.  Since  an  — >  00  as  slowly  as  desired,  moderate  deviations  bridge  the  gap 
between  a  central  limit  approximation  and  a  large  deviations  approximation.  Moderate  deviation 
principles  for  discrete  time  systems  have  been  extensively  studied  in  Mathematical  Statistics  (see 
references  in  [17]),  however  for  continuous  time  stochastic  dynamical  systems  such  principles  are 
less  well  studied  and  none  of  the  prior  results  consider  stochastic  dynamical  systems  with  jumps  or 
infinite  dimensional  models. 

In  this  paper  we  study  moderate  deviation  principles  for  finite  and  infinite  dimensional  SDE 
with  jumps.  For  simplicity  we  consider  only  settings  where  the  noise  is  given  in  terms  of  a  PRM 
and  there  is  no  Brownian  component.  However,  the  more  general  case  where  both  Poisson  and 
Brownian  noises  are  present  can  be  treated  similarly.  In  finite  dimensions,  the  basic  stochastic 
dynamical  system  we  study  takes  the  form 

X£(t.)  =  x0  +  f  b(X£  (s))ds  +  [  sG(X£ (s—),y)N£  1(dy,ds). 

Jo  JXx[0,t] 

Here  b  :  and  G  :  Rd  x  X  4  K1*  are  suitable  coefficients  and  N£  1  is  a  Poisson  random 

measure  on  Xj1  =  X  x  [0,  T]  with  intensity  measure  At,  where  X  is  a  locally  compact 
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Polish  space,  v  is  a  locally  finite  measure  on  X,  A t  is  the  Lebesgue  measure  on  [0,  T]  and  e  >  0  is 
the  scaling  parameter.  Under  conditions  X£  will  converge  in  probability  (in  a  suitable  path  space) 
to  X°  given  as  the  solution  of  the  ODE 

X°(t)  =  b(X°(t))  +  [  G(X°(t),y)u(dy),  X°(0)  =  x0. 

Jx 

The  moderate  deviations  problem  for  {X£}e>o  corresponds  to  studying  asymptotics  of 

(e/a2(e))logP (Y£  <e  ■), 

where  Y£  =  ( X£  —  X°)/a(e )  and  a(e)  —>  0,  e/a2(e)  — »•  0  as  £  — >  0.  In  this  paper  we  establish 
a  moderate  deviations  principle  under  suitable  conditions  on  b  and  G.  We  in  fact  give  a  rather 
general  sufficient  condition  for  a  moderate  deviation  principle  to  hold  for  systems  driven  by  Poisson 
random  measures.  This  sufficient  condition  covers  many  finite  and  infinite  dimensional  models  of 
interest.  A  typical  infinite  dimensional  model  corresponds  to  the  SPDE 

dX£(u,t)  =  (LX£(u,t)  +  (3(X£(u,t,)))dt  +  e  f  G(X£ (u,t—),u,y)N£  1(ds,dy )  (13) 

Jx 

X£(u,  0)  =  x(u),  mGOc 

where  L  is  a  suitable  differential  operator,  O  is  a  bounded  domain  in  Wl  and  the  equation  is 
considered  with  a  suitable  boundary  condition  on  dO.  Here  N£  is  a  PRM  as  above.  The  solution 
of  such  a  SPDE  has  to  be  interpreted  carefully,  since  typically  solutions  for  which  LX£(u ,  t)  can  be 
defined  classically  do  not  exist.  We  follow  the  framework  of  Kallianpur  and  Xiong(1995),  where  the 
solution  space  is  described  as  the  space  of  ROLL  trajectories  with  values  in  the  dual  of  a  suitable 
nuclear  space.  We  establish  a  MDP  for  such  infinite  dimensional  systems  by  verifying  the  sufficient 
condition  given  in  the  work. 

(XXIV)  Long  Time  Results  for  a  Weakly  Interacting  Particle  System  in  Discrete  Time 
[25].  Stochastic  dynamical  systems  that  model  the  evolution  of  a  large  collection  of  weakly  interact¬ 
ing  particles  have  long  been  studied  in  statistical  mechanics,  however  in  recent  years  such  models 
have  been  considered  in  many  other  application  areas  as  well,  some  examples  include,  chemical 
and  biological  systems(  e.g.  biological  aggregation,  chemotactic  response  dynamics),  mathematical 
finance  (e.g.  mean  held  games,  default  clustering  in  large  portfolios),  social  sciences  (e.g.  opinion 
dynamics  models),  communication  systems  etc.  Starting  from  the  work  of  Sznitman(1991)  there 
has  been  an  extensive  body  of  work  that  studies  law  of  large  number  behavior  (Propagation  of 
Chaos),  central  limit  theory  (normal  fluctuations  from  the  mean)  and  large  deviation  principles  for 
such  models.  Ah  of  these  results  concern  the  behavior  of  the  system  over  a  finite  time  horizon.  In 
many  applications  the  time  asymptotic  behavior  of  the  system  is  of  central  concern.  For  example, 
stability  of  a  communication  system,  steady  state  aggregation  and  self  organization  in  biological 
and  chemical  systems,  long  term  consensus  formation  mechanisms  in  opinion  dynamics  modeling, 
particle  based  approximation  methods  for  invariant  measures  all  rely  on  a  careful  analysis  of  the 
time  asymptotic  behavior  of  such  systems.  Such  behavior  for  special  families  of  weakly  interacting 
particle  systems  has  been  considered  by  several  authors  (see  [25]  for  references).  In  the  current 
work  we  consider  a  discrete  time  weakly  interacting  particle  system  and  the  corresponding  nonlin¬ 
ear  Markov  process  in  Md,  described  in  terms  of  a  general  stochastic  evolution  equation.  Denoting 
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by  Xln  =  XfN  the  state  of  the  i-th  particle  (i  =  1, N)  at  time  instant  n,  the  evolution  is  given 
as 

K+i  =  AXln  +  5f(Xl^,eil+1),  i  =  l,..,N,  neN0  (14) 

Here  ^  :=  jj  YliLi  is  the  empirical  measure  of  the  particle  values  at  time  instant  n,  A  is 
a  d  x  d  matrix,  5  is  a  small  parameter,  {eln,i  =  1  n  >  1}  is  an  i.i.d  array  of  valued 

random  variables  with  common  probability  law  6  and  /  :  x  ViW1)  x  is  a  measurable 

function.  Also,  {Xq ,i  =  1,  ...,1V}  are  taken  to  be  exchangeable  with  common  distribution  /jq •  The 
following  nonlinear  Markov  chain  corresponds  to  the  N  — >  oo  limit  of  (14). 

Xn-\-i  —  AXn  A  S f  (Xn,  nn,  €n. )-i ) ,  £(Xn)  —  fin,  n  <E  No.  (15) 

where  jC(X)  denotes  the  probability  distribution  of  a  random  variable  X  with  values  in  some  Polish 
space  S.  Under  conditions  on  /,  6 , 6  and  A  we  study  several  long  time  properties  of  the  IV-particle 
system  and  the  associated  nonlinear  Markov  chain.  Our  starting  point  is  the  evolution  equation 
for  the  law  of  the  nonlinear  Markov  chain  given  by  the  equation 


fln+l  =  T  (nn)-  (16) 

where  T  :  V(M.d)  —$■  V(M.d)  is  defined  as  vk(/j)  =  fiP^. 

We  show  that  under  conditions,  that  include  a  Lipschitz  property  of  /  with  the  Wasserstein- 
l(Wi)  distance  on  the  space  of  probability  measures,  contractivity  of  A  and  5  being  sufficiently 
small,  (16)  has  a  unique  fixed  point  and  starting  from  an  arbitrary  initial  condition  convergence 
to  the  fixed  point  occurs  at  an  exponential  rate.  Using  this  result  we  next  argue  that  under  an 
additional  integrability  condition,  as  N  — >  oo,  the  empirical  measure  /i)}  of  the  IV-particles  at  time 
n  converges  to  the  law  /xn  of  the  nonlinear  Markov  process  at  time  n,  in  the  W}  distance,  in  L1 , 
uniformly  in  n.  This  result  in  particular  shows  that  the  W}  distance  between  ^  and  the  unique 
fixed  point  fioo  of  (16)  converges  to  zero  as  n  — »  oo  and  JV  -»  oo  in  any  order.  This  result  is  key 
in  developing  particle  based  numerical  schemes  for  approximating  the  fixed  point  of  the  evolution 
equation  (16).  We  next  show  that  under  an  irreducibility  condition  on  the  underlying  Markovian 
dynamics  the  unique  invariant  measure  of  the  IV-particle  dynamics  is  ^oo-chaotic,  namely  as 
N  — >•  oo,  the  projection  of  on  the  first  ^-coordinates  converges  to  for  every  k  >  1.  This 
propagation  of  chaos  property  all  the  way  to  n  =  oo  crucially  relies  on  the  uniform  in  time  conver¬ 
gence  of  to  /loo.  The  next  three  results  study  the  rate  of  this  uniform  convergence  by  developing 
suitable  probability  concentration  estimates.  The  first  result,  under  an  assumption  of  polynomial 
moments  on  the  initial  data  and  noise  sequence  establishes  a  corresponding  uniform  in  time  poly¬ 
nomial  concentration  bound.  The  proof  relies  on  an  idea  of  restricting  measures  to  a  compact 
set  and  estimates  on  metric  entropy  introduced  in  Bolley  et.  al.(2007).  The  basic  idea  is  to  first 
obtain  a  concentration  bound  for  the  Wi  distance  between  the  truncated  law  and  its  corresponding 
empirical  law  in  a  compact  ball  of  radius  R  along  with  an  estimate  on  the  contribution  from  the 
region  outside  the  ball  and  finally  optimize  suitably  over  R.  The  last  two  results  are  concerned  with 
exponential  concentration.  These  impose  much  stronger  integrability  conditions  on  the  problem 
data.  The  first  considers  the  setting  where  the  initial  random  variables  form  a  general  exchangeable 

i 

sequence  and  gives  a  concentration  bound  with  an  exponential  decay  rate  of  N  d+2 .  The  second 
result  uses  exponential  concentration  estimates  for  empirical  measures  of  i.i.d.  sequences  based  on 
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recent  results  on  transportation  inequalities  In  this  case  the  concentration  bound  gives  an  expo¬ 
nential  decay  rate  of  order  N. 


(XXV)  On  Uniform  Positivity  of  Transition  Densities  of  Small  Noise  Constrained 
Diffusions  [13]. 

Diffusions  in  polyhedral  domains  have  been  extensively  studied  in  the  heavy  traffic  limit  theory 
for  stochastic  processing  networks  (see  references  in  [13]).  In  a  recent  work  [6]  small  noise  asymp¬ 
totics  for  a  general  family  of  such  constrained  diffusions  have  been  studied.  The  precise  setting 
there  is  as  follows.  Let  G  C  be  convex  polyhedral  cone  with  a  nonempty  interior  with  the  vertex 
at  origin  given  as  the  intersection  of  half  spaces  Gi,i  =  1,  2, ...,  N.  Let  n*  be  the  unit  inward  normal 
vector  associated  with  G{  via  the  relation 

Gi  =  {.x  G  Rfc  :  (x,rii)  >  0}. 

We  will  denote  the  set  {x  G  dG  :  (x,rii)  =  0}  by  Ft.  With  each  face  Ft  we  associate  a  unit  vector 
di  such  that  (di,n,i)  >  0.  This  vector  defines  the  direction  of  constraint  associated  with  the  face 
Fi.  Roughly  speaking,  a  constrained  diffusion  considered  in  this  work  is  a  process  that  evolves 
infinitesimally  as  a  diffusion  in  and  is  instantaneously  pushed  back  using  the  oblique  reflection 
direction  di  upon  reaching  the  face  Fi.  Formally,  such  a  process  can  be  represented  as  a  solution 
of  a  stochastic  integral  equation  of  the  form 

X£(t)  =  r(x  +  f  b(X£(s))ds  +  e  a(X£(s))dW(s)^J  (t),  (17) 

where  T  is  the  Skorohod  map  taking  trajectories  with  values  in  to  those  with  values  in  G, 
consistent  with  the  constraint  vectors  {di,i  =  1,  •  •  •  N}.  Under  certain  regularity  assumptions  on 
the  Skorohod  map  and  the  usual  Lipschitz  conditions  on  the  coefficients  b  and  a,  the  above  integral 
equation  has  a  unique  pathwise  solution.  One  of  the  main  results  of  [6]  is  an  ‘exponential  leveling’ 
property  of  exit  times  from  bounded  domains  for  such  small  noise  diffusions.  Such  results  for 
diffusions  in  have  been  obtained  in  Day (1982)  which  is  concerned  with  asymptotics  of  Dirichlet 
problems  in  bounded  domains  associated  with  diffusions  with  infinitesimal  generator  of  the  form 

£e/(x)  =  ^Tr(a{x)D2f(x)a(x)tr)  +  b(x)  ■  V/(s),  /  G  Cf  (Mfc).  (18) 

Here  a(x)tr  is  the  transpose  of  the  matrix  a(x).  The  precise  result  of  Day  is  as  follows.  Denote 
by  Z£  the  diffusion  process  governed  by  the  generator  C£  and  initial  distribution  Sx.  Let  B  be  a 
bounded  domain  in  and  K  be  an  arbitrary  compact  subset  of  B.  Suppose  that  all  solutions  of  the 
ODE  £  =  b(f  )  with  x  =  £(0)  G  B  converge,  without  leaving  B ,  to  a  single  linearly  asymptotically 
stable  critical  point  in  B.  Then,  with  suitable  conditions  on  the  coefficients  of  the  diffusion,  for  all 
bounded  measurable  / 

sup  | E  ( f{Zex{r£x )))  -  E  ( f(Zy(r£ )))  | 
x,yeK 

converges  to  0  at  an  exponential  rate.  Here,  r|  =  infjf  :  Zf(t)  G  Bc}.  This  property  is  a  statement 
on  the  long  time  behavior  of  the  diffusion  and  says  that  although,  as  e  — >  0,  the  exit  time  of 
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the  process  from  the  domain  approaches  oo,  the  expected  values  of  functionals  of  exit  location, 
corresponding  to  distinct  initial  conditions,  coalesce  asymptotically,  at  an  exponential  rate.  The 
key  ingredient  in  the  proof  is  the  gradient  estimate 

sup  |Vu£(.x)|  <  ce^1/2, 

xeK 

where  u£  is  the  solution  of  the  Dirichlet  problem 

Ceu£(x)  =  0,  x  G  B 
u(x)  =  f(x),  x  G  dB. 

Diffusions  of  interest  in  the  current  work  are  constrained  to  take  values  in  domains  with  corners 
and  where  the  constraining  mechanism  is  governed  by  an  oblique  reflection  field  that  changes 
discontinuously  from  one  face  of  the  boundary  to  another.  To  the  best  of  our  knowledge  there  are 
no  regularity  (e.g.  C1  solutions)  results  known  for  the  associated  partial  differential  equations (PDE) 
with  oblique  reflecting  boundary  condition.  In  view  of  this,  a  probabilistic  approach  for  the  study  of 
exponential  leveling  property  for  such  constrained  settings,  that  ‘almost’  bypasses  all  PDE  estimates 
was  developed  in  [6].  The  main  step  in  the  proof  is  the  construction  of  certain  (uniform  in  e) 
Lyapunov  functions  under  a  suitable  stability  condition  which  are  then  used  to  construct  a  coupling 
of  the  processes  X£ ,  X£  with  explicit  uniform  estimates  on  exponential  moments  of  time  to  coupling. 
The  key  ingredient  in  this  coupling  construction  is  a,  uniform  in  e,  minorization  condition  on 
transition  densities  of  the  reflected  diffusions.  The  paper  [6]  gave  one  simple  example  with  constant 
drift  and  diffusion  coefficients  where  such  a  condition  is  satisfied.  However  the  question  of  when 
such  a  minorization  property  is  available  was  left  as  an  open  problem. 

The  objective  of  this  work  is  to  answer  this  question  and  give  general  conditions  under  which 
the  minorization  property  holds.  The  main  result  of  the  paper  shows  that  under  mild  conditions 
the  desired  minorization  statement  holds.  This  result,  together  with  results  of  [6]  then  gives  general 
sufficient  conditions  for  an  exponential  leveling  property  to  hold  for  a  broad  family  of  constrained 
diffusion  processes.  The  proof  at  one  step  crucially  relies  on  an  interior  lower  bound  estimate  for 
the  Dirichlet  heat  kernel  of  Cs  over  bounded  domains  (Riahi(2005)).  In  this  sense  the  proofs  are 
not  fully  probabilistic. 

(XXVI)  Dynamic  Scheduling  for  Markov  Modulated  Single-server  Multiclass  Queue¬ 
ing  Systems  in  Heavy  Traffic  [20]. 

Heavy  traffic  formulation  provides  tractable  approximations  for  complex  queueing  systems  that 
capture  broad  qualitative  features  of  the  networks.  While  most  works  in  the  literature  deal  with 
fixed  (or  internal  network-state-dependent)  rates,  with  the  advent  of  modern  wireless  networks  there 
is  an  explosion  of  research  on  models  for  networks  where  external  factors  such  as  meteorological 
variables  (temperature,  humidity,  etc)  can  affect  the  transmission  rates  to  and  from  the  servers. 
The  current  paper  deals  with  the  simplest  such  setting  of  a  multiclass  network,  namely  a  single 
server  multiclass  queueing  system  in  heavy  traffic,  where  the  arrival  and  services  fluctuate  according 
to  an  environment  process.  In  the  classical  constant  rate  setting  this  model  has  been  studied  in  van 
Mieghem(1995),  in  the  conventional  heavy  traffic  regime.  The  queueing  system  consists  of  a  single 
server  which  can  process  K  different  classes  of  jobs  ( K  >  1).  The  K  classes  represent  different  types 
of  jobs  (voice  traffic,  data  traffic  etc.),  and  they  have  different  arrival  and  service  rate  functions. 


(19) 


(20) 
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We  assume  that  these  functions  are  modulated  by  a  finite  state  Markov  process  that  represents  the 
background  environment.  The  arrival  and  service  rates  satisfy  a  suitable  heavy  traffic  assumption 
which,  loosely  speaking,  says  that  the  network  capacity  and  service  requirements  are  balanced  in 
the  long  run.  We  consider  a  scheduling  control  problem,  where  the  controller  decides  (dynamically, 
at  each  time  point  t  >  0)  which  class  of  jobs  should  the  server  process  so  as  to  minimize  an  infinite 
horizon  discounted  cost  function,  which  involves  a  linear  holding  cost  per  job  per  unit  time  for 
each  of  the  K  different  classes.  The  jump  rates  of  the  modulating  Markov  process  are  modeled  by 
a  scaling  parameter  u.  The  value  of  u  can  vary  from  negative  to  positive  -  the  larger  the  value 
of  z/,  the  faster  the  environment  changes  -  and  we  consider  three  distinct  regimes  for  u.  We  show 
that  under  different  values  of  zq  with  suitably  scaling,  queue  length  processes  stabilize  leading  to 
different  diffusion  approximations.  Main  result  of  the  paper  shows  that,  in  each  of  these  regimes, 
an  asymptotically  optimal  scheduling  policy  is  a  variation  of  the  classical  and  intuitively  appealing 
c/x-rule.  Classical  c/x- rule  is  a  simple  priority  policy,  where  the  priority  is  always  given  to  the  class 
(of  nonempty  queue)  with  the  highest  c/x-value  where  c  and  /x  represent  the  holding  cost  and  the 
service  rate  of  that  class,  respectively.  It  is  well  known  that  c/x- rule  is  optimal  in  many  cases, 
see  van  Mieghem(1995)  and  references  therein.  In  our  model,  the  arrival  rates  change  randomly 
according  to  the  background  Markov  process,  and  it  is  far  from  clear  if  the  classical  c/x-rule  would 
be  optimal.  In  addition,  in  one  of  the  regimes  we  consider,  the  service  rates  are  allowed  to  be 
modulated  by  the  random  environment  as  well,  which  means  that  the  order  of  the  c/x-v alues  keeps 
changing  according  to  the  environment.  In  particular,  in  situations  where  the  environment  process 
is  not  directly  observable  and  one  only  knows  (or  can  estimate)  its  statistical  properties,  the  classical 
c/x-rule  cannot  be  implemented.  We  propose  a  modified  c/x*- rule,  where  /x*  represents  the  average 
service  rate  of  each  class  and  the  average  is  taken  with  respect  to  the  stationary  distribution  of 
the  environment  process.  Under  the  c/x*- rule,  the  server  always  processes  jobs  from  the  nonempty 
queue  with  the  highest  c/x*-v alue.  We  show  that,  under  an  appropriate  heavy  traffic  condition  and 
a  suitable  scaling,  the  c/x*-rule  is  asymptotically  optimal  for  the  chosen  cost  functional  for  this 
environment-dependent  queueing  model,  in  each  of  the  three  regimes  for  u. 

(XXVII)  Large  deviations  for  Multidimensional  State-Dependent  Shot  Noise  Pro¬ 
cesses  [24]. 

Shot  noise  processes  are  used  in  applied  probability  to  model  a  variety  of  physical  systems  in, 
for  example,  teletraffic  theory,  insurance  and  risk  theory  and  in  the  engineering  sciences.  In  this 
work  we  prove  a  large  deviation  principle  for  the  sample-paths  of  a  general  class  of  multidimensional 
state-dependent  Poisson  shot  noise  processes.  The  result  covers  previously  known  large  deviation 
results  for  one  dimensional  state-independent  shot  noise  processes  with  light  tails.  We  use  the  weak 
convergence  approach  to  large  deviations,  which  reduces  the  proof  to  establishing  the  appropriate 
convergence  of  certain  controlled  versions  of  the  original  processes  together  with  relevant  results  on 
existence  and  uniqueness. 

(XXVIII)  Individual  confidence  intervals  for  true  solutions  to  stochastic  variational 
inequalities  [33].  Stochastic  variational  inequalities  (SVI)  provide  a  means  for  modeling  various 
optimization  and  equilibrium  problems  where  data  are  subject  to  uncertainty.  Often  it  is  necessary 
to  estimate  the  true  SVI  solution  by  the  solution  of  a  sample  average  approximation  (SAA)  problem. 
This  paper  proposes  three  methods  for  building  confidence  intervals  for  components  of  the  true 
solution,  and  those  intervals  are  computable  from  a  single  SAA  solution.  The  first  two  methods  use 
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an  “indirect  approach”  that  requires  initially  computing  asymptotically  exact  confidence  intervals 
for  the  solution  to  the  normal  map  formulation  of  the  SVI.  The  third  method  directly  constructs 
confidence  intervals  for  the  true  SVI  solution;  intervals  produced  with  this  method  meet  a  minimum 
specified  level  of  confidence  in  the  same  situations  for  which  the  first  two  methods  are  applicable. 
We  justify  the  three  methods  theoretically  with  weak  convergence  results,  discuss  how  to  implement 
these  methods,  and  test  their  performance  using  two  numerical  examples. 

Ph.D.  Dissertation  Abstract  of  Xin  Liu.  Applications  arising  from  computer,  telecommuni¬ 
cations,  and  manufacturing  systems  lead  to  many  challenging  problems  in  the  simulation,  stability, 
control,  and  design  of  stochastic  models  of  networks.  The  networks  are  usually  too  complex  to  be 
analyzed  directly  and  thus  one  seeks  suitable  approximate  models.  One  class  of  such  approxima¬ 
tions  are  diffusion  models  that  can  be  rigorously  justified  when  networks  are  operating  in  heavy 
traffic,  i.e.,  when  the  network  capacity  is  roughly  balanced  with  network  load.  We  study  stochastic 
networks  with  time  varying  arrival  and  service  rates  and  rout-  ing  structure.  Time  variations  are 
governed,  in  addition  to  the  state  of  the  system,  by  two  independent  finite  state  Markov  processes 
X  and  Y .  Transition  times  of  X  are  significantly  smaller  than  the  typical  inter  arrival  and  pro¬ 
cessing  times  whereas  the  reverse  is  true  for  the  Markov  process  Y .  We  first  establish  a  diffusion 
approximation  for  such  multiscale  queueing  networks  in  heavy  traffic.  The  result  shows  that,  under 
appropriate  heavy  traffic  conditions,  properly  normalized  queue  length  processes  converge  weakly 
to  a  Markov  modulated  reflected  diffusion  process.  More  precisely,  the  limit  process  is  a  reflected 
diffusion  with  drift  and  diffusion  coefficients  that  are  functions  of  the  state  process,  the  invariant 
distribution  of  X  and  a  finite  state  Markov  process  which  is  independent  of  the  driving  Brownian 
motion.  We  then  study  the  stability  properties  of  such  Markov  modulated  reflected  diffusion  pro¬ 
cesses  and  establish  positive  recurrence  and  geometric  ergodicity  properties  under  suitable  stability 
conditions.  As  consequences,  we  obtain  results  on  the  moment  generating  function  of  the  invariant 
probability  measure,  uniform  in  time  moment  estimates  and  functional  central  limit  results  for 
such  processes.  We  also  study  relationship  between  invariant  measures  of  the  Markov  modulated 
constrained  diffusion  processes  and  that  of  the  underlying  queueing  network.  It  is  shown  that, 
under  suitable  heavy  traffic  stability  conditions,  the  invariant  probability  measure  of  the  queueing 
network  converges  to  that  of  the  corresponding  Markov  modulated  reflected  diffusion  model.  The 
last  part  of  this  dissertation  focuses  on  ergodic  control  problems  for  discrete  time  controlled  Markov 
chains  with  a  locally  compact  state  space  and  a  compact  action  space  under  suitable  stability,  irre- 
ducibility  and  Feller  continuity  conditions.  We  introduce  a  flexible  family  of  controls,  called  action 
time  sharing  (ATS)  policies,  associated  with  a  given  continuous  stationary  Markov  control.  It  is 
shown  that  the  long  term  average  cost  for  such  a  control  policy,  for  a  broad  range  of  one  stage  cost 
functions,  is  the  same  as  that  for  the  associated  stationary  Markov  policy.  Through  examples  we 
illustrate  the  use  of  such  ATS  policies  for  parameter  estimations  and  adaptive  control  problems. 


Ph.D.  Dissertation  Abstract  of  Dominik  Reinhold.  This  dissertation  is  composed  of  two 
parts,  a  theoretical  part,  in  which  certain  asymptotic  properties  of  near  critical  branching  processes 
are  studied,  and  an  applied  part,  consisting  of  statistical  analysis  of  cell  growth  data.  First,  near 
critical  single  type  Bienaym  e-Galton- Watson  (BGW)  processes  are  con-  sidered.  It  is  shown  that, 
under  appropriate  conditions,  Yaglorn  distributions  of  suitably  scaled  BGW  processes  converge 
to  that  of  the  corresponding  diffusion  approximation.  Convergences  of  stationary  distributions 
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for  Q-processes  and  models  with  immigration  to  the  corresponding  distributions  of  the  associated 
diffusion  approximations  are  estab-  lished  as  well.  Moreover,  convergence  of  Yaglorn  distributions 
of  suitably  scaled  multitype  subcritical  BGW  processes  to  that  of  the  associated  diffusion  model  is 
established.  Next,  near  critical  catalyst-reactant  branching  processes  with  controlled  immigration 
are  considered.  The  catalyst  population  evolves  according  to  a  classical  continuous  time  branching 
process,  while  the  reactant  population  evolves  according  to  a  branching  process  whose  branching 
rate  is  proportional  to  the  total  mass  of  the  catalyst.  Immigration  takes  place  exactly  when  the 
catalyst  population  falls  below  a  certain  threshold,  in  which  case  the  population  is  instantaneously 
replenished  to  the  threshold  to  ensure  a  certain  activity  level.  A  diffusion  limit  theorem  for  the 
scaled  processes  is  established,  in  which  the  catalyst  limit  is  a  reflected  diffusion  with  reflection  at 
1,  and  the  reactant  limit  is  a  diffusion  with  coefficients  depending  on  the  reactant.  The  driving 
Brownian  motions  in  the  two  diffusions  are  independent. 

Stochastic  averaging  under  fast  catalyst  dynamics  are  considered  next.  In  the  setting  where  both 
catalyst  and  reactant  evolve  according  to  the  above  described  (reflected)  diffusions,  but  where  the 
evolution  of  the  catalyst  is  accelerated  by  a  factor  of  n,  we  establish  a  scaling  limit  theorem,  as  n  — > 
oo,  in  which  the  reactant  process  is  asyrnp-  totically  described  through  a  one  dimensional  SDE  with 
coefficients  depending  on  the  invariant  distribution  of  the  catalyst  reflected  diffusion.  Convergence 
of  the  stationary  distribution  of  the  scaled  catalyst  branching  process  (with  immigration)  to  that 
of  the  limit  reflected  diffusion  is  established  as  well.  Finally,  results  from  a  collaborative  proof-of- 
principle  study,  relating  cell  growth  to  the  stiffness  of  the  surrounding  environment,  are  presented. 

Ph.D.  Dissertation  Abstract  of  Jiang  Chen.  This  dissertation  studies  large  deviations  prob¬ 
lems  for  stochastic  dynamical  systems.  First,  a  family  of  Stochastic  Partial  Differential  Equations 
(SPDE)  driven  by  a  Poisson  Random  Measure  (PRM)  that  are  motivated  by  problems  of  chemi¬ 
cal/pollutant  dispersal  are  considered.  A  Large  Deviation  Principle  (LDP)  for  the  long  time  profile 
of  the  chemical  concentration  using  techniques  based  on  variational  representations  for  nonnegative 
functionals  of  general  PRM  is  established.  Second,  a  LDP  for  small  Poisson  noise  perturbations  of 
a  general  class  of  deterministic  infinite  dimensional  models  is  studied.  SPDEs  driven  by  PRM  have 
been  proposed  as  models  for  many  different  physical  systems.  The  approach  taken  here,  which  is 
based  on  variational  representations,  reduces  the  proof  of  the  LDP  to  establishing  basic  qualitative 
properties  for  controlled  analogues  of  the  underlying  stochastic  system.  Third,  stochastic  systems 
with  two  time  scales  are  studied.  Such  multiscale  systems  arise  in  many  applications  in  engineering, 
operations  research  and  biological  and  physical  sciences.  The  models  considered  in  this  dissertation 
are  usually  referred  to  as  systems  with  full  dependence,  which  refers  to  the  feature  that  the  coeffi¬ 
cients  of  both  the  slow  and  the  fast  processes  depend  on  both  variables.  A  LDP  for  such  systems 
with  degenerate  diffusion  coefficients  is  established.  The  last  part  of  this  dissertation  focuses  on 
numerical  schemes  for  computing  invariant  measures  of  reflected  diffusions.  Reflected  diffusions  in 
polyhedral  domains  are  commonly  used  as  approximate  models  for  stochastic  processing  networks 
in  heavy  traffic.  Stationary  distributions  of  such  models  give  useful  information  on  the  steady  state 
performance  of  the  corresponding  stochastic  networks  and  thus  it  is  important  to  develop  reliable 
and  efficient  algorithms  for  numerical  computation  of  such  distributions.  A  Monte-Carlo  scheme 
based  on  an  Euler  type  discretization  is  proposed  and  analyzed.  An  almost  sure  consistency  result 
that  proves  the  convergence  of  the  appropriately  weighted  empirical  measures  constructed  from  the 
simulated  discretized  reflected  diffusion  to  the  true  diffusion  model  is  established.  Rates  of  conver¬ 
gence  are  also  obtained  for  certain  class  of  test  functions.  Some  numerical  examples  are  presented 
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to  illustrate  the  applicability  of  this  approach. 


Ph.D.  Dissertation  Abstract  of  Xuan  Wang.  This  dissertation  consists  of  two  parts.  The 
first  part  studies  phase  transition  of  a  class  of  dynamical  random  graph  processes,  that  evolve  via 
the  addition  of  new  edges  in  a  manner  that  incorporates  both  randomness  as  well  as  limited  choice. 
As  the  density  of  edges  increases,  the  graphs  display  a  phase  transition  from  the  subcritical  regime, 
where  all  components  are  small,  to  the  supercritical  regime,  where  a  giant  component  emerges. 
The  goal  is  to  understand  the  behavior  at  criticality.  First,  the  simplest  model  of  this  kind,  namely 
the  Bohman-Frieze  process  is  considered.  It  is  shown  that  the  stochastic  process  of  component 
sizes,  in  the  critical  window  for  the  Bohman-Frieze  process  after  proper  scaling,  converges  to  the 
standard  multiplicative  coalescent.  Next,  a  more  general  family  of  dynamical  random  graph  models 
is  studied,  namely,  the  bounded-size-rule  processes.  A  useful  upper  bound  on  the  size  of  the  largest 
component  in  the  barely  subcritical  regime  is  established.  Next,  this  upper  bound  is  used  to  study 
both  sizes  and  surplus  of  the  components  of  the  bounded-size-rule  processes  in  the  critical  window. 
In  order  to  describe  the  joint  evolution  of  sizes  and  surplus,  the  augmented  multiplicative  coales¬ 
cent  is  introduced.  The  main  result  shows  that  the  vector  of  suitably  scaled  component  sizes  and 
surplus  converges  in  distribution  to  the  augmented  multiplicative  coalescent.  In  the  second  part  of 
this  dissertation,  a  large  deviation  problem  related  to  the  configuration  model  with  a  given  degree 
distribution  is  studied.  A  random  walk  associated  with  the  depth-hrst-exploration  of  the  random 
graph  constructed  from  the  configuration  model  is  introduced.  The  large  deviation  principle  of  this 
random  walk  is  studied  using  weak  convergence  techniques.  Some  large  deviation  bounds  on  the 
probabilities  related  to  the  sizes  of  the  largest  component  are  proved. 
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